AWS S3

Before setting up S3 connectivity in Dataworkz, necessary permission should have been configured in AWS for the same

AWS IAM Policy and User Creation

Dataworkz recommends creating an IAM policy and user for Dataworkz to access the S3 bucket.

Once the policy is attached to the bucket/user and AWS's security credentials are generated, these credentials need to be configured in Dataworkz to access objects in S3.

IAM Policy

From the AWS management console, navigate to the IAM section. Choose Account Settings from the left pane.

Check the Security Token Service Regions list and confirm that the region for your account is active as shown below.

IAM User

To create a new AWS IAM user, choose the “Users” option in the left pane and click on “Add users”.

Specify the user name and save the same.

Click the newly created user from the list and select the “Security credentials” tab.

Scroll down to the “Access keys” section and click “Create access key”

Choose “Other” in the next section since Dataworkz only has permission to access objects in a specific bucket.

Create the access key and click “Download .csv file” for configuring it in Dataworkz.

S3 Bucket

Create a new S3 bucket that is managed using IAM policies.

There are 2 options to add permissions for Dataworkz to access the buckets.

• Attaching a policy to the bucket (Bucket Permissions)

OR

• Attaching a policy to the user (User Permissions)

Bucket Permissions

Select the bucket and goto the “Permissions” tab and edit the bucket policy.

Give all the permissions (read & write) on the bucket and the encapsulating directory (see the image above). Edit the policy json to include the Principal of the user created for the purpose (see section “IAM User”).

User Permissions

Goto IAM -> User

Select the user and goto the “Permissions” section and select “Add Permission”. Select the option to attach policies directly. Select AmazonS3FullAccess and save the same. This would give user access to all the S3 resources for read, write & list.

Configuring S3 connector in Dataworkz

• Login to Dataworkz Application

• Go to Configuration -> Cloud Data Platforms -> S3

• Click the + icon to add a new S3 connection

• Give the Storage a name

• Add the Key (your S3 user security access key)

• Add the Value (your S3 user security key value)

• Select the storage base path (url path to the storage location)

• Choose the region of choice

• Test the connection

• Once you have a successful connection, Save the configuration