Google Cloud Storage

Polytomic can connect to Google Cloud Storage using a service account key:

Go to https://console.cloud.google.com/apis/credentials/serviceaccountkey to create a new service account.
Select the Cloud Storage Storage Transfer Admin role:

Create and save a JSON service account key file by selecting the newly created service account Manage Details
→ Keys → Add Key → Create new key → JSON → Create. A .json file will be downloaded to your computer.
In Polytomic, go to Connections → Add Connection → Google Cloud Storage .

Enter your Google Cloud Storage Project ID.
Upload your JSON key file from step 3 in Polytomic's Service Account Key field.
Choose the Google Cloud Storage destination Bucket.
Click Save.

Malformed CSVs

Polytomic ignores malformed CSVs. This means that, when using Polytomic to sync from CSV files on Google Cloud Storage, you may see CSVs missing from the source file list if they exhibit malformations like the following:

Empty column names.
Duplicate column names.
Rows with fewer columns than the number of columns in the file header.

If you don't see an expected CSV file listed in Polytomic, please check it for these issues.

Concatenating multiple CSV or JSON files into one SQL table

When using Polytomic's Bulk Sync functionality to sync from Google Cloud Storage, you have the option of having Polytomic concatenate all CSV or JSON files in your bucket into one table in your data warehouse. You can do by turning on the Files are time-based snapshots setting in your connection configuration:

Once you turn on this setting, you will also need to specify these settings:

Collection name: This will be the name of the resulting SQL table in your data warehouse.
File format: Instructs Polytomic to either concatenate all CSV files in the bucket or all JSON files.
Skip first lines: If your CSVs have lines at the top that need to be skipped before getting to the headers for your data, you can specify the number of lines Polytomic should skip in this field.