Setup Instructions for Amazon S3 Destination
Learn how to effectively set up an Amazon S3 database and seamlessly integrate it with Daton for enhanced data management and integration capabilities.
Setup Instructions for Amazon S3 Destination
Learn how to effectively set up an Amazon S3 database and seamlessly integrate it with Daton for enhanced data management and integration capabilities.
Setting up access to your Amazon S3 data warehouse is not very complicated and just takes a couple of simple steps. If you don't have an existing account on AWS, you can sign up for a free tier here.
Follow the steps below to set it up for yourself, if you are stuck somewhere, feel free to contact out support team at support@sarasanalytics.com.
Pre-Requisites
- Access to Amazon Web Services
- Access to Daton to configure the destination
- Permissions to manage S3 buckets in AWS
Setup on Amazon S3
Task 1: Create an S3 Bucket
1. Use the search functionality on your AWS console to easily navigate to the S3 Setup console.
2. Create a Storage Bucket or Use an existing Storage Bucket depending on your requirement.
3. Enter the bucket name and select the appropriate options to define the bucket parameters according to your organization preference.
Task 2: Granting Bucket Permissions
1. One the bucket is created, grant permissions to Daton to create and manage files in this bucket.
2. Edit the bucket policy and add the following JSON to the policy after changing the bucket name to reflect the bucket where you want data loaded.
{
"Version": "2012-10-17",
"Id": "",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::214669062314:role/Warehouse"
]
},
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::<YOUR_BUCKET_NAME>",
"arn:aws:s3:::<YOUR_BUCKET_NAME>/*"
]
}
]
}
The steps laid out above are a pre-requisite before you start configuring S3 as a destination in Daton.
Setup on Daton
1. Log in to your Daton account and navigate to the Destinations page.
2. Click the Add New Destination button and select Amazon S3 from the list of options.
3. Enter an integration name to identify the warehouse in Daton.
4. Enter the bucket name that is going to host the files replicated by Daton
5. Enter the Object Key Prefix.
6. Select the region in which you have created the bucket.
Currently, we support CSVs, NDJSON, Parquet and JSON for S3 buckets. If you need support for any other format, do write to us at support@sarasanalytics.com
Optionally you can add the additional keys to Object key prefix by selecting optional key elements field.
- The time zone used is UTC.
- They key would have the following format if all the optional elements are selected:
[object_key_prefix]/[integration_name]/[tablename]/[year]/[month]/[day]/[hour]/[timestamp.filetype]
If only a combination of elements are selected then the order of optional elements in the key will be in descending order of the time period.
7. Click on Submit to complete the setup.
