AWS S3

Configure AWS IAM permissions and create an S3 connector in Dataworkz for cloud object store access.

Before setting up S3 connectivity in Dataworkz, configure the necessary permissions in AWS.

AWS IAM Policy and User Creation

Dataworkz recommends creating a dedicated IAM policy and user to access the S3 bucket. After attaching the policy to the bucket or user and generating AWS security credentials, configure those credentials in Dataworkz to access objects in S3.

IAM Policy

In the AWS Management Console, navigate to the IAM section. Select Account Settings from the left pane.

Review the Security Token Service Regions list and confirm that the region for your account is active.

IAM User

To create a new AWS IAM user, select Users in the left pane and click Add users.

Enter a user name and save it.

Click the newly created user from the list and select the Security credentials tab.

Scroll down to the Access keys section and click Create access key.

Select Other in the next section, since Dataworkz only accesses objects in a specific bucket.

Create the access key and click Download .csv file to save the credentials for use in Dataworkz.

S3 Bucket

Create a new S3 bucket managed by IAM policies.

There are two options for granting Dataworkz access to the bucket:

• Attach a policy to the bucket (Bucket Permissions)

• Attach a policy to the user (User Permissions)

Bucket Permissions

Select the bucket, go to the Permissions tab, and edit the bucket policy.

Grant read and write permissions on the bucket and its enclosing directory. Edit the policy JSON to include the Principal of the IAM user created for Dataworkz (see IAM User).

User Permissions

Go to IAM > Users.

Select the user, go to the Permissions section, and click Add Permission. Choose Attach policies directly. Select AmazonS3FullAccess and save. This grants the user read, write, and list access to all S3 resources.

Configuring S3 Connector in Dataworkz

1. Log in to the Dataworkz application.

2. Go to Configuration > Cloud Data Platforms > S3.

S3 — list of configured connectors

1. Click + Add new configuration.

Add New S3 configuration form

1. Enter a Storage name for this connector.

2. Select the Access Type (default: Key).

3. Enter the Access Key (your AWS IAM user access key ID).

4. Enter the Secret (your AWS IAM user secret access key).

5. Enter the Storage base path in the format s3://bucket-name/path.

6. Select the Region where your S3 bucket is hosted.

7. Click Test Connection.

8. Once the connection succeeds, click Save.