Federation using AD

Demo Video

Client Tool

This demo will use a browser to access the AWS Console. You will also need to a SQL Client configured. This demo will use EC2 ADFS AMI. The screenshots for this demo will leverage the EC2 AMI. Be sure to complete the BI Tool setup before proceeding.

Before you Begin

Ensure you have access to the AWS Console and an existing Redshift cluster with permissions to modify it. Capture the following parameters from the launched CloudFormation template as you will use them in the demo.

https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks?filteringText=&filteringStatus=active&viewNested=true&hideStacks=false
  • RedshiftClusterEndpoint
  • InfrastructureAccount

Challenge

Rachel (Security Engineer) has been tasked to enable federated single sign-on for the business users and setup various groups and privileges for the database objects. This will help provide a seamless user experience for the user, while also helping centralized management and governance of permission and authentication for hundreds of users at Red Imports LLC. Rachel will need to implement the following workflow to allow users to login to Redshift using SSO.

  1. User logs in using JDBC/ODBC SQL Client
  2. The IdP authenticates using Corp Username/Password and returns a SAML assertion.
  3. Calls STS to assume role with SAML
  4. STS Returns Temp Credentials
  5. Uses Temp Credentials to get Temp cluster credentials
  6. Connect to Amazon Redshift using temp credentials

Part 1 ADFS Server Setup

  1. ADFS SSO Setup WorkFlow
  2. Setup domain controller and AD FS
  3. Install AD FS
  4. Configure federation in AD FS
  5. Setup claim rules
  6. Configure custom claim rules
  7. Test SAML federation using web browser

Part 2 Amazon Redshift Setup

  1. Setup SQL Client
  2. Connect to Amazon Redshift and create Schema and Tables
  3. Grant privileges to database groups
  4. Connect using federated authentication and test user access

Part 1: ADFS Server Setup

Setup domain controller and AD FS

Complete the preparation steps in SSO Considerations before proceeding.

SayDo Show
Once you have the domain controller EC2 server up and running, now you can Use an RDP client from your desktop to connect the EC2 instance using the administrator user and password from previous step. Use an RDP client (Microsoft Remote Desktop) from your desktop to connect the EC2 instance using the administrator user and password from previous step.
Create test users and groups in AD FS using the Windows “Active Directory Users and Computers” application or using command prompt by entering the following code (this walkthrough creates the user Bob, which you use again in Part 2): Go to windows command prompt in the Windows EC2 instance which you connected using RDP. Execute the commands below to create the users and groups:

dsadd user "cn=Bob,cn=Users,dc=adfsredshift,dc=com" -samid bob -upn bob@adfsredshift.com -email bob@adfsredshift.com -fn Bob -ln Smith -display "Bob Smith" -disabled no -pwd "YourOwnStrongPassword"
dsadd group "cn=AWS-Production, cn=Users, dc=adfsredshift, dc=com" -members "cn=Bob, cn=Users, dc=adfsredshift, dc=com"
dsadd group "cn=AWS-Dev, cn=Users, dc=adfsredshift, dc=com" -members "cn=Bob, cn=Users, dc=adfsredshift, dc=com"
dsadd user "cn=ADFSSVC,cn=Users, dc=adfsredshift,dc=com" -display "ADFS Service" -disabled no -pwd " YourOwnStrongPassword “
dsadd group "cn=RSDB-marketing, cn=Users, dc=adfsredshift, dc=com" -members "cn=Bob, cn=Users, dc=adfsredshift, dc=com"

Install AD FS

SayDo Show
On the same Windows 2016 EC2 instance, install the AD FS 2016 role using PowerShell From command line execute the command to install the AD FS application

PS C:\Users\Administrator> Install-windowsfeature adfs-federation –IncludeManagementTools

To create the self-signed certificate, use PowerShell to install and configure the IIS From windows command line execute the following command

PS C:\Users\Administrator> Install-WindowsFeature -name Web-Server -IncludeManagementTools
  1. In the Amazon EC2 Windows server, under Administrative Tools, open IIS.
  2. Choose the IIS server
  3. Choose Server Certificates
  4. Under Actions, choose Create Self-Signed Certificate
  5. Enter AD FSCertificate
  6. Choose Ok

Configure federation in AD FS

SayDo Show
Configure AD FS federation

On the Windows 2016 EC2 instance

  1. Open Server Manager
  2. Choose the flag icon
  3. Choose Configure the federation server on this server.

You can ignore the warning message to promote this server to domain controller. This doesn’t interfere with the setup because as the AD FS is working at this point and the server is already a domain controller.

Select Create the first federation server in a federation server farm.

  1. Choose Next
  2. On the SSL Certificate drop-down menu
  3. Choose the self-signed certificate you created during IIS setup. The Federation Service Name gets auto-populated.
  4. For Federation Service Display Name, enter ADFSRedshiftDemo
  5. Choose Next
  6. Confirm the names to ensure that it’s valid
  7. Choose Ok
Enter the password for the service account – ADFSSVC that you created earlier, in previous step

Enter the password for the service account – ADFSSVC that you created earlier, in previous step. Choose Next
Choose Next two more times. Review and run the prerequisite checks and choose Configure

  1. Choose Next two more times
  2. Review and run the prerequisite checks and choose Configure You can see the installation progress and AD FS installation starts. You may receive a warning message, but you can ignore it for this walkthrough. You also need to restart the Windows 2016 EC2 instance.

After you RDP back into the Windows 2016 EC2 instance, download the federation metadata from AD FS.

To capture the hostname, under Windows Administrative Tools

  1. Choose AD FS Manager
  2. Choose Edit Federation Service Properties
  3. Copy the federation service name, for example YourInstanceHostname.adfsredshift.com
  4. Install Google Chrome on the EC2 instance via RDP by entering the following code in the Windows PowerShell command prompt:
	$LocalTempDir = $env:TEMP; $ChromeInstaller = "ChromeInstaller.exe"; (new-object System.Net.WebClient).DownloadFile('http://dl.google.com/chrome/install/375.126/chrome_installer.exe', "$LocalTempDir\$ChromeInstaller"); & "$LocalTempDir\$ChromeInstaller" /silent /install; $Process2Monitor =  "ChromeInstaller"; Do { $ProcessesFound = Get-Process | ?{$Process2Monitor -contains $_.Name} | Select-Object -ExpandProperty Name; If ($ProcessesFound) { "Still running: $($ProcessesFound -join ', ')" | Write-Host; Start-Sleep -Seconds 2 } else { rm "$LocalTempDir\$ChromeInstaller" -ErrorAction SilentlyContinue -Verbose } } Until (!$ProcessesFound)

Download AD FS metadata

SayDo Show
Internet Explorer on the EC2 instance generated by the CloudFormation template is, by default, in Enhanced Security configuration, so may not be available to use.

You may choose to install any other browser or disable Enhanced Security for Internet Explorer.

From Google Chrome, enter the following URL with the hostname from the previous step:

https://<<YourInstanceHostname.adfsredshift.com>>/FederationMetadata/2007-06/FederationMetadata.xml

If you receive an error message, choose the advance icon and then choose Proceed to download the file.

Copy the FederationMetadata.xml file from the RDP download location and save it to your local desktop to use later.

Configuring AWS as the relying party with ADFS

SayDo Show
To configure AWS as the relying party, you first need to create the SAML provider and federation role, and create and attach IAM policies. To create the SAML provider, complete the following steps:

  1. On the IAM console, choose Identity Provider
  2. Choose Create provider
  3. For Provider type, choose SAML
  4. For Provider name, enter ADFZ
  5. For Choose File, choose the FederationMetadata.xml you downloaded earlier
  6. Choose Create You should see the provider as in the following screenshot. You should continue to use the name ADFZ for the SAML IdP because this name carries over in other parts of the post and is referred to in the claim rules

Creating a SAML federation role

SayDo Show
You can now create a SAML federation role. On the IAM console, choose Roles:

  1. Choose Create role.
  2. Choose SAML 2.0 Federation.
  3. For SAML provider, enter ADFZ
  4. For Attribute, choose SAML:aud
  5. For Value, enter https://signin.aws.amazon.com/saml

Complete creation of the role with the below steps

  1. Choose Next.
  2. Choose any permissions you need for the role (for this walkthrough, choose AmazonRedshiftFullAccess).
  3. In a production environment, it’s advised to use a role with restricted access as required.
  4. Choose Next twice.
  5. For Role name, choose ADFZ-Dev.
  6. Repeat the previous steps to create another role called ADFZ-Production.

Creating an IAM policy

SayDo Show
For the ADFZ-Production role, you now create an IAM policy. On the IAM console, choose Policies. Choose Create policy.

On the JSON tab, enter the following policy: The GetClusterCredentials Redshift API with AWS credentials to get the temporary database credentials. This API creates the user and adds it to a specified DB group, so you also need to provide redshift:CreateClusterUser and redshift:JoinGroup permissions to the IAM role. See the following code:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "redshift:ListSchemas",
                "redshift:DescribeQuery",
                "redshift:ListDatabases",
                "redshift:DescribeClusters",
                "redshift:ExecuteQuery",
                "redshift:GetClusterCredentials",
                "redshift:FetchResults",
                "redshift:DescribeTable",
                "redshift:CreateClusterUser",
                "redshift:ListTables",
                "redshift:CancelQuery"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "redshift:JoinGroup",
            "Resource": "*"
        }
    ]
}

Ideally, development and production IAM roles are in two different accounts. The Resource section here is set to be open with *, but in a production environment, you should specify the exact resource you want to control with the policy.

  1. Enter a policy name, for example redshift-marketing.
  2. Choose Create policy.
  3. Return to the ADFZ-Production role.
  4. Choose Attach policies.
  5. Choose the policy you created.

Configuring AWS as the relying party

SayDo Show
To complete configuring AWS as the relying party, go back to RDP on Windows 2016 EC2 instance. In the AD FS management application, choose Add Relying Party Trust.

Complete the steps shown

  1. Choose Claims aware.
  2. Choose Start.
  3. On the next page, for Data Source, enter the standard URL as https://signin.aws.amazon.com/static/saml-metadata.xml for importing data about the relying party published online or on a local network option.
  4. Choose Next.
  5. Enter a display name, for example, Amazon Web Services.
  6. Choose Next.
  7. For Access Control Policy, choose Permit everyone.
  8. Choose Next with default choices until you see the Finish page.
  9. Choose Close.

Configure claim rules

SayDo Show
AD FS uses a claims-based identity model and issues a token that contains a set of claims as part of its function as federation services. Claim rules govern the decision with regards to claims that AD FS issues. Claim rules and all server configuration data are stored in the AD FS configuration database. AD FS makes issuance decisions that are based on claims and other contextual information by taking one set of claims as input, performing transformations, and returning a different set of claims as output. For this walkthrough, you set up two standard claim rules and three custom claim rules. Configuring standard claim rules After you choose Close in the previous step, you are redirected to a window to define the claim rules. If this doesn’t happen, you can return to the AD FS management console on the Windows 2016 EC2 instance, choose Relying Party Trusts, and choose Edit Claim Issuance Policy. To add a rule for NameID, complete the following steps:

  1. Choose Choose Rule Type.
  2. For the claim rule template, choose Transform an Incoming Claim.
  3. Choose Next.
  4. For Claim rule name, enter NameId.
  5. For Incoming claim type, choose Windows account name.
  6. For Outgoing claim type, enter Name ID.
  7. For Outgoing Name ID format, choose Persistent identifier.
  8. Select Pass through all claim values.
  9. Choose Finish.
You now add another rule for RoleSessionName.
  1. Choose Choose Rule Type.
  2. For Claim rule template, choose Send LDAP Attributes as Claims.
  3. For Claim rule name, enter RoleSessionName.
  4. For Attribute store, choose Active Directory.
  5. For LDAP Attribute, choose E-Mail Addresses.
  6. For Outgoing Claim Type, enter https://aws.amazon.com/SAML/Attributes/RoleSessionName.
  7. Choose Finish.

Configure custom claim rules

SayDo Show
By using a custom claim rule, you can create rules with more complex logic than a standard rule template. You use custom rules to send claims based on values that are extracted from a custom attribute store for AD groups, DB groups, and roles.

To add a custom rule for your AD group, complete the following steps:

  1. Choose Choose Rule Type.
  2. For Claim rule template, choose Send Claims Using a Custom Rule.
  3. Choose Next.
  4. For Claims rule name, enter Get AD Groups.
  5. Add the following custom rule:
c:[Type == "http://schemas.microsoft.com/ws/2008/06/identity/claims/windowsaccountname", Issuer == "AD AUTHORITY"] => add(store = "Active Directory", types = ("http://temp/variable"), query = ";tokenGroups;{0}", param = c.Value);
  1. Choose Finish.
  2. To add a custom rule for roles, you replace the string with your account ID using the temporary variable you created earlier. This step takes all the groups from the temporary variable from previous step and replaces AWS- with an Amazon Resource Name (ARN) with your account ID and the prefix for the roles created in AWS. The list is then sent as the path in the following code.

Follow the previous steps to create the rule Roles.

  1. Enter the following custom rule, replacing your own account ID and role prefix:
c:[Type == "http://temp/variable", Value =~ "(?i)^AWS-"] => issue(Type = "https://aws.amazon.com/SAML/Attributes/Role", Value = RegExReplace(c.Value, "AWS-", "arn:aws:iam::<InfrastructureAccount>:saml-provider/ADFZ,arn:aws:iam::<InfrastructureAccount>:role/ADFZ-"));

You now add a third custom rule for the database group DBGroups mapping of an AD group RSDB-marketing to a database group marketing, which you create later in Amazon Redshift in Part2. Follow the previous steps to create the rule DBGroups. Enter the following custom rule:

c:[Type == "http://temp/variable", Value =~ "(?i)^RSDB-"]=> issue(Type = "https://redshift.amazon.com/SAML/Attributes/DbGroups", Value = RegExReplace(c.Value, "RSDB-", ""));

Choose Finish. In this claim rule, you take an AD group starting with the name RSDB- and remove the prefix. The remaining string (marketing), matches the database group name. This allows the user after SAML authentication to join the database group with same name, which is only valid during the session window. This allows the user to access the resources allowed by the database group marketing, which is defined at the database level.

Confirm the five rules are correct and choose Ok.

Test SAML federation using web browser

SayDo Show
Before starting your test, run the following code using PowerShell: Run the following code using PowerShell:

Set-AdfsProperties -EnableIdpInitiatedSignonPage $true

To test your federation, complete the following steps:

  1. Open Google Chrome on Windows Server 2016 through RDP.
  2. Enter the following link: https://«YourInstanceHostname.adfsredshift.com»/adfs/ls/IdpInitiatedSignOn.aspx.
  3. Sign in as bob@adfsredshift.com and use the password set for the AD user Bob.
  4. For Select a role, select ADFZ-Dev.
  5. Choose Sign in.

Part 2 Amazon Redshift Setup

This demo walks you through setting up an Amazon Redshift cluster and federating database user authentication with AWS Identity and Access Management (IAM) and Amazon Redshift. You use the user you set up in your AD in Part 1 (Bob) to authenticate using AD FS and control access to database objects based on the group the user is assigned to.

You also learn how to set up SQL Client (SQL Workbench/J) and an Amazon Redshift JDBC driver to connect to a database cluster using the database user authentication method and the AD FS federated authentication method.

Prerequisites and Architecture

This demo assumes the steps in Part 1 is completed. If you have launched the demo Cloudformation template, the setup involved in Part 1 will already be configured In Part 2, you will use the Demo Amazon Redshift cluster for federated authentication with AD FS to connect from a JDBC SQL client tool. You follow a detailed step-by-step process of using Amazon Redshift database authentication to simplify the credential management of database users and reuse what you may already have on premises. You can also integrate Amazon Redshift group-level privileges with federation.

Setup SQL Client

SayDo Show
Before you get started, download the latest Amazon Redshift JDBC driver with AWS SDK. You use this for JDBC driver-based authentication in SQL Workbench/J.

  1. Download and install SQL Workbench/J. For instructions, see Connect to your cluster by using SQL Workbench/J.
  2. Open SQL Workbench/J and choose Manage Drivers.
  3. Choose Amazon Redshift.
  4. For Library, browse to the JDBC driver you downloaded.
  5. Press Ok.

Connect to Amazon Redshift and create Schema and Tables

SayDo Show
To set up federated access, you take a two-step approach: connecting with the superuser and connecting using federated authentication. For this walkthrough, you create database objects, groups, and users and assign proper privileges to the groups on the database objects they are allowed to access. You should receive a successful connection message, as in the following screenshot.

Choose New Connection. Construct a JDBC connection URL for the database standard user: jdbc:redshift://«redshiftclusterendpoint».redshift.amazonaws.com:5439/dev?ssl=true&UID=username&PWD=password

You can find the value for redshiftclusterendpoint on the Amazon Redshift console, under Properties, Endpoint. The username and password are the values you provided when you created the cluster.

Choose Test.

Connect as awsuser (a superuser). From SQL Workbench/J, enter the following commands to set up the following environment:

  1. Create two database groups
  2. Create two schemas
  3. Create two tables in each schema
  4. Insert sample data into the tables, and test that data exists by querying the tables
CREATE GROUP sales;
CREATE GROUP marketing;

CREATE SCHEMA sales;
CREATE SCHEMA marketing;

CREATE TABLE IF NOT EXISTS marketing.employee
(
	n_empkey INTEGER   
	,n_name CHAR(25)   
	,n_regionkey INTEGER   
	,n_comment VARCHAR(152)   
)
DISTSTYLE AUTO
 SORTKEY (n_empkey);

CREATE TABLE IF NOT EXISTS sales.employee_sales
(
	n_empkey INTEGER   
	,n_name CHAR(25)   
	,n_regionkey INTEGER   
	,n_comment VARCHAR(152)   
)
DISTSTYLE AUTO
 SORTKEY (n_empkey);

INSERT INTO marketing.employee
VALUES(1, 'Bob', 0, 'Marketing');

INSERT INTO sales.employee_sales
VALUES(1, 'John', 0, 'Sales');

Select * from marketing.employee;

Select * from sales.employee_sales;

Grant privileges to database groups

SayDo Show
You can now set up appropriate privileges for the sales and marketing groups. Groups are collections of users who are all granted privileges associated with the group. You can use groups to assign privileges by job function. For example, you can create different groups for sales, administration, and support and give the users in each group the appropriate access to the data they require for their work. You can grant or revoke privileges at the group level, and those changes apply to all members of the group, except for superusers.

Enter the following SQL queries to grant access to all tables in the sales schema to the sales group and access to all tables in the marketing schema to the marketing group:

ALTER DEFAULT PRIVILEGES IN SCHEMA sales;
GRANT SELECT on TABLES to GROUP sales;
GRANT USAGE on SCHEMA sales to GROUP sales;
GRANT SELECT on ALL TABLES in SCHEMA sales to GROUP sales;

ALTER DEFAULT PRIVILEGES IN SCHEMA marketing;
GRANT SELECT on TABLES to GROUP marketing;
GRANT USAGE on SCHEMA marketing to GROUP marketing;
GRANT SELECT on ALL TABLES in SCHEMA marketing to GROUP marketing;

Connect using federated authentication and test user access

SayDo Show
You can pass the URL information in the connection profile two different ways:

Specify the Amazon Redshift cluster endpoint with the port (for this use case, the driver doesn’t need to run the DescribeClusters API because everything is already available): jdbc:redshift:iam//«your redshift cluster endpoint»/dev

Specify the cluster ID and Region (for this use case, the driver calls the DescribeClusters API and Get-Cluster-Credentials as additional steps): jdbc:redshift:iam://«cluster id:region»/dev

The rest of the parameters are specified in the Extended Properties configuration.

For this walkthrough, you use the second method.

  1. Create a new connection profile in SQL Workbench/J.
  2. Choose Extended Properties.
The extended properties section allows you to defind the IdP host and the IAM role to be used for authentication Setting AutoCreate= True creates the AD user Bob in the Amazon Redshift database the first time without password settings. The authentication is temporary using the SAML configuration you completed in Part 1. You can also set the AutoCreate property with IdP configuration. For more information, see JDBC and ODBC Options for Creating Database User Credentials. Setup the below values in the extended properties section

Provide the following values and Choose Test. You should see a successful connection, and can start using SQL Workbench/J to query Amazon Redshift cluster as user Bob. The message indicates Connection to endpoint successful

  1. idp_port : 443
  2. plugin_name : com.amazon.redshift.plugin.AdfsCredentialsProvider
  3. ssl_insecure : true
  4. idp_host : The host name for the AD domain, for example, windows2016.adfsredshift.com
  5. preferred_role – The ARN of the SAML role you created in Part 1, for example, arn:aws:iam::<InfrastructureAccount>:role/ADFZ-Production (replace <InfrastructureAccount> with your AWS Account)
  6. AutoCreatetrue

With this connection profile, run the following query to test SAML authentication and query privilege control: The output shows that the user bob@adfsredshift.com was authenticated using AD FS. The user also joined the marketing group as enforced by the AD FS DbGroups claim rule and the policy associated with the ADFZ-Production role, which the user assumes during this session.

  1. Enter the below select statement and see the results; You should see that the user session was logged in with “marketing” group
select * from stv_sessions;

Now you can test the group level access control by running queries on the two tables in different schemas. First, execute a query on a table in marketing schema “marketing.employee” The output shows that AD user Bob is part of the AD group RSDB-marketing, which was mapped to the DB group marketing. This database group had select access to the schema marketing and all tables in that schema. Therefore, the user could successfully query the table through SQL Workbench/J.

  1. To run the query against the marketing schema, enter the following code:
select * from marketing.employee;

Next, execute a query on a table in sales schema sales.employee The output shows that Bob is only part of the AD group RSDB-marketing. Due to the way the claim rule is set up, Bob doesn’t have access to the database group sales, and therefore the query returns with a permission denied error. To run a query against the sales schema, enter the following code:

select * from sales.employee_sales;

Before you Leave

If you want to avoid incurring charges, delete the CloudFormation stack on the console.

If you are done using your cluster, please think about deleting the CFN stack or to avoid having to pay for unused resources do these tasks:

  • pause your Redshift Cluster
  • stop the Oracle database
  • stop the DMS task
  • delete the EC2 created