AWS S3
Last updated
Last updated
Before setting up S3 connectivity in Dataworkz, necessary permission should have been configured in AWS for the same
Dataworkz recommends creating an IAM policy and user for Dataworkz to access the S3 bucket.
Once the policy is attached to the bucket/user and AWS's security credentials are generated, these credentials need to be configured in Dataworkz to access objects in S3.
From the AWS management console, navigate to the IAM section. Choose Account Settings from the left pane.
Check the Security Token Service Regions list and confirm that the region for your account is active as shown below.
To create a new AWS IAM user, choose the “Users” option in the left pane and click on “Add users”.
Specify the user name and save the same.
Click the newly created user from the list and select the “Security credentials” tab.
Scroll down to the “Access keys” section and click “Create access key”
Choose “Other” in the next section since Dataworkz only has permission to access objects in a specific bucket.
Create the access key and click “Download .csv file” for configuring it in Dataworkz.
Create a new S3 bucket that is managed using IAM policies.
There are 2 options to add permissions for Dataworkz to access the buckets.
OR
Select the bucket and goto the “Permissions” tab and edit the bucket policy.
Goto IAM -> User
Select the user and goto the “Permissions” section and select “Add Permission”. Select the option to attach policies directly. Select AmazonS3FullAccess and save the same. This would give user access to all the S3 resources for read, write & list.
Login to Dataworkz Application
Go to Configuration -> Cloud Data Platforms -> S3
Click the + icon to add a new S3 connection
Give the Storage a name
Add the Key (your S3 user security access key)
Add the Value (your S3 user security key value)
Select the storage base path (url path to the storage location)
Choose the region of choice
Test the connection
Once you have a successful connection, Save the configuration
Attaching a policy to the bucket ()
Attaching a policy to the user ()
Give all the permissions (read & write) on the bucket and the encapsulating directory (see the image above). Edit the policy json to include the Principal of the user created for the purpose (see section “”).
