Export Storage: Bring your own Bucket
Learn how to leverage your own storage buckets for exports.
The GrowthLoop platform stores some data in storage buckets to route data to marketing platforms or power other features like debug logs.
Your team can choose to use either:
- A GrowthLoop-managed storage bucket, or
- Your own bucket (Bring Your Own Bucket โ BYOB)
What Data Is Stored?
GrowthLoop temporarily stages audience or segment membership data in a staging storage bucket to allow our export service to sync members to marketing platforms.
After routing:
- The data is deleted
- Buckets have a TTL of 7 days
Bring Your Own Bucket (BYOB)
Your team can store all at-rest data in a private bucket managed in your own infrastructure.
Currently supported:
- Amazon S3 (AWS)
- Google Cloud Storage (GCS)
Amazon S3 Setup
1. Create a New S3 Bucket
- Create a new S3 bucket in your AWS account
- Ensure you:
- Block public access
- Configure an object lifecycle policy
2. Create an IAM Policy
- In AWS Console โ Select IAM
- Select Policies
- Click Create Policy
- Choose JSON
- Paste the following policy (replace
<bucket>with your bucket name):
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:GetObjectVersion"
],
"Resource": "arn:aws:s3:::<bucket>/exports/*"
},
{
"Effect": "Allow",
"Action": [
"s3:GetBucketLocation",
"s3:GetBucketAcl"
],
"Resource": "arn:aws:s3:::<bucket>"
},
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": "arn:aws:s3:::<bucket>",
"Condition": {
"StringLike": {
"s3:prefix": [
"exports/*"
]
}
}
}
]
}- Enter a policy name
- Click Create Policy
IAM Permissions Explained
| Permission | Details |
|---|---|
| s3:PutObject | Upload files to bucket |
| s3:GetObject | Retrieve objects |
| s3:GetObjectVersion | Access specific object versions |
| s3:ListBucket | List objects |
| s3:GetBucketLocation | Retrieve bucket region |
| s3:GetBucketAcl | View bucket ACL |
3. Create an IAM Role
- Go to IAM โ Roles
- Select Create Role
- Choose AWS Account
- Keep This Account selected
- Enable Require external ID
- Enter placeholder
0000
- Attach the policy created above
- Name and create the role
- Open the role โ verify the policy is attached
- Edit the Trust Relationship
Replace the trust policy with:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::667705482227:user/growthloop-exports"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "Gr0wthloop4RtG8-2"
}
}
}
]
}Record the Role ARN from the role summary page.
Please provide the role ARN and bucket ARN to your GrowthLoop Solutions Architect.
Additional Steps for Snowflake Integration
1. Create Storage Integration in Snowflake
USE ROLE ACCOUNTADMIN;
CREATE STORAGE INTEGRATION FLYWHEEL_APP_AWS
TYPE = EXTERNAL_STAGE
STORAGE_PROVIDER = 'S3'
ENABLED = TRUE
STORAGE_AWS_ROLE_ARN = '<iam_role>'
STORAGE_ALLOWED_LOCATIONS = ('s3://<bucket>/exports/');
GRANT USAGE ON INTEGRATION FLYWHEEL_APP_AWS TO ROLE <growthloop_role>;Where:
<iam_role>= ARN of IAM role created above<bucket>= S3 bucket name<growthloop_role>= Role used by GrowthLoop Snowflake service account
2. Create External Stage
USE ROLE ACCOUNTADMIN;
CREATE OR REPLACE STAGE flywheel_external_stage
URL = 's3://<bucket>/exports/<org_id>/'
STORAGE_INTEGRATION = FLYWHEEL_APP_AWS;
GRANT OWNERSHIP ON INTEGRATION FLYWHEEL_APP_AWS TO ROLE <growthloop_role>;Note: The Storage Integration must be created in the FLYWHEEL_SYSTEM schema.
Where:
<org_id>โ provided by your GrowthLoop Solutions Architect
3. Retrieve Snowflake IAM User
DESC INTEGRATION GROWTHLOOP_APP_AWS;Record:
STORAGE_AWS_IAM_USER_ARNSTORAGE_AWS_EXTERNAL_ID
4. Update AWS Trust Policy for Snowflake
Edit the IAM Role trust relationship and add:
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"AWS": "<snowflake_user_arn>"
},
"Action": "sts:AssumeRole",
"Condition": {
"StringEquals": {
"sts:ExternalId": "<snowflake_external_id>"
}
}
}Where:
<snowflake_user_arn>=STORAGE_AWS_IAM_USER_ARN<snowflake_external_id>=STORAGE_AWS_EXTERNAL_ID
5. Create IAM User for GrowthLoop
- Go to IAM โ Users
- Select Create User
- Assign the IAM role created above
- Generate:
- Access key
- Secret access key
- Save credentials
Use these credentials in:
https://docs.growthloop.com/docs/bring-your-own-bucket#setup-external-storage-in-growthloop
Google Cloud Storage (GCS) Setup
1. Create GCS Bucket
- Create a new bucket
- Ensure:
- Block public access
- Configure a lifecycle policy / TTL
2. Create Service Account
Grant the following permissions:
| Permission | Details |
|---|---|
| storage.objects.list | List objects |
| storage.objects.create | Create objects |
| storage.objects.get | Retrieve objects |
| storage.buckets.get | View bucket metadata |
3. Create Credentials
- Go to IAM & Admin โ Service Accounts
- Select your service account
- Click Keys โ Add Key โ Create New Key
- Select JSON
- Download and save as
credentials.json
You will use this file in:
https://docs.growthloop.com/docs/bring-your-own-bucket#setup-external-storage-in-growthloop
Setup External Storage in GrowthLoop
- Go to Settings โ Organization
- Navigate to the Storage tab
- Select your bucket provider
- Enter credentials
- Click Save
GrowthLoop will now use your secure bucket for staging data.
If you have questions, contact:
Updated 19 days ago