Table filters

About

When syncing data into your data warehouse or cloud storage, you may want to filter some of it out before it gets written to your storage system.

For example, you may be syncing a large event stream into your data warehouse where you only need a couple of event types rather than all of them.

Polytomic's bulk syncs thus come with filters you can set on your incoming dataset. They apply to text, numbers, and dates and are processed in-flight (that is, before the data is written to your storage).

How to set filters

Once you select tables to sync, the Add filters button will be enabled:

Clicking on it will pop up a drawer where you can select a field from any of the tables you are syncing:

Selecting a field will let you set a filter condition of your choosing:

In the above screenshot, we will only sync Accounts whose Business Type matches startup or SMB.

You can set as many filters as you like by clicking the + button in the screenshot above, in which case they will be combined with an AND condition: