Google BigQuery

Use GrowthLoop audiences for table exports to Google BigQuery.

Connect this destination to export audience membership as rows in a table you choose, for use in Google BigQuery.

This destination writes to your Google BigQuery project and is a good fit when you want to:

  • Write audience rows in BigQuery rather than a paid-media or messaging API
  • Match contacts on a unique identifier

This maps to GrowthLoop Audiences and Journeys:

Use it forAudiencesJourneys
Writing audience membership to a Google BigQuery table✓✓

To connect BigQuery as your GrowthLoop warehouse source, see Sources. To land CSV files in a bucket, use Google Cloud Storage.

Prerequisites

Make sure you have:

  • Access to a BigQuery data warehouse.
  • A Google Cloud service account with the Editor role on the project, and its key saved as a JSON file.

To create the service account, you need serviceAccountAdmin and BigQuery Admin permissions. If you are not sure you have them, check with your Google Cloud administrator.

If you already have the JSON key, skip to Set up Google BigQuery as a destination.

Create a service account

  1. In Google Cloud, search for Service Accounts.
Google Cloud search bar used to find Service Accounts
  1. Click Create Service Account.
Create Service Account button on the Service Accounts page
  1. Enter the service account details, then click Create and Continue.
FieldDescription
Service Account NameA descriptive name. Separate words with dashes
Service Account IDGoogle Cloud fills this in. An email address for the service account appears below it
Service Account DescriptionA short description of why you created the account
Create service account form with name, ID, and description
  1. Grant the service account the Editor role on the project, then click Done.
Service account access step with the Editor role selected

Create a JSON key

JSON keys let GrowthLoop use the service account. Save the file in a secure place. The key is not recoverable. If you lose it, create another. You can have up to 10 keys per service account.

You need administrative privilege over keys for the project. If you get a permissions error, ask your Google Cloud administrator to grant key creation access or create the key and send you the JSON.

  1. In Google Cloud, search for Service Accounts and open that page.
Google Cloud search results for Service Accounts
  1. Open the service account you created.
Service account selected from the Service Accounts list
  1. Open Keys. Click Add Key → Create New Key.
Service account Keys tab with Create New Key
  1. Select JSON and click Create. The key file downloads to your computer.
Create private key dialog with JSON selected

Set up Google BigQuery as a destination

  1. In GrowthLoop, click the Destinations tab on the left sidebar.
  2. Click New Destination in the upper right corner.
  3. Search for BigQuery under Data Warehouse and click Add BigQuery.
  4. Enter the required information, then click Create.
FieldDescription
Destination NameName this destination in GrowthLoop. Use a name so your team recognizes its purpose, like "Google BigQuery Audiences".
Sync FrequencySet the default export frequency (for example, hourly or daily). You can override this setting for individual exports.
DatasetThe BigQuery dataset where GrowthLoop should create the table
CredentialsThe entire contents of your service account JSON key file
Columns to ExportOptional. Comma-separated warehouse column names. If you leave this blank, GrowthLoop exports all columns
Prefix for Personalization FieldsOptional. Prefix applied to personalization field names. Defaults to fw. Leave blank for no prefix. If you set one, it must start with a letter and use up to 5 letters or numbers

GrowthLoop validates the connection. Google BigQuery then appears in your list of connected destinations.

Export to Google BigQuery

From within GrowthLoop, export from an audience or from a journey Destination Node. See Export an audience or Build a journey if you need more information on these steps.

After building or selecting an audience, click Export, then search for your BigQuery destination. In a journey, add a Destination Node under Datawarehouse and select BigQuery.

Journey Destination Node with export name, match fields, and personalization signals

Map identifiers

Map Unique Identifier. This destination requires a unique identifier for each row.

You can also map custom attributes from the audience or from the journey as additional fields.

Configure the export

This is where you set the campaign type and schedule for audience exports. On a journey Destination Node, you set timing on the journey canvas.

SettingAudience exportJourney Destination Node
Export NameDefaults to the audience name.Required. Enter a name so you can find the export in GrowthLoop.
Table NameRequired. Letters, numbers, and underscores only.Required. Letters, numbers, and underscores only.
Campaign typeChoose a one-time or ongoing export.The journey schedule controls when the journey runs and contacts are sent to Google BigQuery.
Export Inclusion CriteriaAll audience members for a full refresh, or Newly added audience members for additions and removals since the last export.The journey sends contacts who reach the Destination Node.
ScheduleSet frequency, start date, end date, and days of the week for ongoing exports.Users reach this node when the journey runs. Add a Delay Node or set Criteria Evaluation to control timing.
Personalization FieldsExtra columns for this export. By default, GrowthLoop sends the destination Columns to Export.Extra columns for this export.

Export behavior

This is what GrowthLoop writes when you export an audience.

BehaviorWhat happens
What GrowthLoop writesRows in the BigQuery table you named. Direction is GrowthLoop → Google BigQuery.
When it runsGrowthLoop sends the export as a batch on the next successful run. An export that starts Now runs within about 15 minutes. Journeys send exports on demand when users reach the Destination Node.
TableGrowthLoop creates the table in the Dataset you selected during setup, using the Table Name on the export.
Match fieldUnique Identifier is required.
Columns to ExportIf you set this on the destination, GrowthLoop limits the table to those columns unless you add personalization fields.
Personalization prefixPersonalization field names use the prefix you set on the destination, or fw if you left the default.
One-time exportsGrowthLoop sends current membership once.
Ongoing exportsGrowthLoop sends updated membership on each scheduled run.

Confirm the export was successful

When the run finishes, confirm you can use the data in Google BigQuery.

  • Audience export: Open the dataset you selected during setup and look up the table named on the export.
  • Journey Destination Node: Open that dataset and look up the table named on the export.

In GrowthLoop, review counts on the audience Exports tab, or open the Activity tab on the journey Destination Node. See Export an audience or Monitor a journey for more information.

Congrats on successfully exporting to Google BigQuery!

Troubleshooting

These are the issues we see most often. You can always email us at [email protected] for help.

ProblemCauseResolution
Connection fails to validateThe JSON key is incomplete, or the service account cannot write to the projectPaste the entire JSON key. Confirm the service account has the Editor role
You cannot create a service accountYour user lacks serviceAccountAdmin or BigQuery AdminAsk a Google Cloud administrator to grant those roles or create the account for you
You cannot create a JSON keyYour user cannot manage keysAsk a Google Cloud administrator to grant key access or create the key for you
Table Name is rejectedThe name uses characters other than letters, numbers, or underscores, or it is already in useUse only letters, numbers, and underscores, and choose a unique table name
The table is missingYou are looking in the wrong dataset, or Table Name does not match what you entered on the exportOpen the Dataset from destination setup and look for the export Table Name
Expected columns are missingColumns to Export or Personalization Fields omit those columnsAdd the columns on the destination or on the export

What’s Next

Export an audience after you connect a destination

Did this page help you?