Changelog

rel2023.01.18.01

Fixes

Reduce scopes requested for FB Ad Audience connections

rel2023.01.18

Fixes

Optimized Snowflake schema caching

rel2023.01.17

Features

Added support for auto-completion of SQL & SOQL queries in Query Runner.
Added support for syncing Front data.
Added support for syncing to Front Contacts.

Fixes

Columns beginning with _ are now allowed for BigQuery.
Force resyncing an update-only model sync to BigQuery truncated the table before starting the sync.
Reduce Salesforce API consumption for bulk syncs.
PostgreSQL connections made via SSH tunnel were previously incompatible with replication.

rel2023.01.10

Features

Added support for syncing to additional Netsuite objects:
- Non-inventory Items for Purchase
- Other Charge Items for Purchase
- Service Items for Purchase
Added support for BigQuery tables backed by Google Drive files.

Fixes

Updated Affinity rate limiting to support their RPM limit.
Visiting a sync page after a single test sync had executed could result in "No such sync" error in rare circumstances.
Improved healthcheck performance.
Entering an email address for the SSO signin domain would result in an error; we now trim the address to its domain.

rel2023.01.03.02

Fixes

Update Salesforce integration with Enhanced Domain support.
Fields selected from related objects in SOQL models were missing in some circumstances.

rel2023.01.03

Features

Added support for additional Netsuite models:
- Journal entries
- Non-inventory purchase items
- Other charge purchase items
- Service purchase items
- Vendor bills
Added support for Harmonic People Saved Search models.
Improved Affinity integration
- Added support for Affinity People models.
- Fields sourced from Affinity enrichment partners are marked as readonly.
Improved Freshdesk integration
- Added support for Contact, Company, and Ticket models.
- Added support for use as bulk sync source.
Added support for using Google Workspace data as a model.

Fixes

Empty query parameters in API connections could cause errors.
Append syncs to Google Sheets could incorrectly detect the first empty row, resulting in blank rows.
Display AWS User ARN for S3 connections.
Syncs to Redshift could error if the storage bucket included a prefix.
Reduced size of Docker image.
Enabled rate-limit retries for Stripe integration.
Postgres connections using replication were left open under some circumstances.

rel2022.12.20.02

Additional diagnostics for Athena syncs

rel2022.12.20

Features

Added support for Stripe Payment Method US Bank Account models.
Added support for associating Affinity People with Organizations using the Organization name.
Added support for UserVoice connections.
Added support for Vanilla Forums connections.

Fixes

Improved performance when loading Model Syncs.
Bulk syncs could remain in running state after all schemas failed under some conditions.
Improved type detection for Google Sheets columns.
Harmonic School URN was not correctly exposed for People Education models.
Append syncs to Google Sheets could incorrectly detect the first empty row, resulting in blank rows.
Record count for Facebook Audience syncs were incorrect under some conditions.
Sync execution errors that could not be associated with a record were counted but not displayed in the sync history.

rel2022.12.13

Features

Added support for using Vanilla Forums Discussions as a model.

Fixes

Improved performance when loading sync list.
Added support for using Affinity organization name in an identity mapping.
BigQuery result sets greater than 10GB could result in sync errors.
Null BigQuery arrays could be improperly encoded when syncing to Snowflake.

rel2022.12.06.01

Fixes

Fix cases where we overzealously delegate SQL joins to source systems

rel2022.12.06

Features

New Stripe Bulk Sync objects: Events, Refunds, Balance Transactions, Accounts, Transfers
Intercom Bulk Sync support: Tags, Conversations, Admins
Netsuite Source/Destination support: Vendor objects
Harmonic connections can now enrich data with Apollo.
Parition keys can now be set in the bulk sync interface for BigQuery
The Query Runner buffer will now remember the state from the last time you used it.
BigQuery connections can now override their ProjectID for cross-account access.

Fixes

Fix scheduling of continuous bulk syncs
Snowflake Views will now show up in Query Runner
Google Sheets cell values are now truncated to 50k characters
Better localization on some Salesforce GraphQL requests

rel2022.11.29.01

Fixes

Order of fields in sync configurations could be non-deterministic.

rel2022.11.29

Features

Added support for LIKE/NOT LIKE string conditions.

Fixes

The order of fields in CSV files created during syncs could be non-deterministic.
It was not possible to remove authentication from an API connection after adding it.
Use of CloudSQL connections could result in elevated CPU and memory load.
Objects were moved to a cancelled state in bulk syncs under some circumstances.

rel2022.11.22

Features

Added support for Netsuite Invoice Items as a model.

Fixes

Creating a sync to a recently dropped table could result in an error.
Errors during bulk sync startup could result in a cancellation rather than failure.
Lookup fields for user-defined associations in Hubspot could be incorrectly labeled.

rel2022.11.15.01

Features

Google Analytics event metadata includes organization identifier.

Fixes

Athena direct syncs to S3 could fail when results and destination buckets were in different regions.

rel2022.11.15

Features

Added support for using MySQL, MS SQL as bulk sync sources.

Fixes

Add query runner database exclusion list (#3891)
Date picker for model filter values displayed with Monday as start of week.
Creating a model from a MongoDB aggregation could fail in some circumstances.
Types were not correctly detected for some nested field values in MongoDB models.
Improved performance of loading the syncs page.

rel2022.11.10

Features

__pt_rowid is now included in bulk sync sources (when available).

Fixes

More durable API sources with longer timeouts and better errors messages.
Fixed a bug where syncs could miss an exectuion between deployments
Clicking refresh on the bulk sync schema list will cause Polytomic to
refresh its list of schemas directly from the source.
Fixed a memory consumption affecting single-model syncs
Affinity jobs now have additional protection against running write jobs twice
Various bugfixes to Postgres Replication

rel2022.11.08

Fixes

Whitespace in URL paths could cause HTTP API integrations to fail

rel2022.11.04

Features

Syncs using multiple Athena models will delegate joins to Athena
Expose highlight categories as fields for Harmonic models
Added support for creating Lob postcards via sync

Fixes

Netsuite sandbox account identifiers could be specified in invalid format
#ERROR values in Google Sheets cells caused value errors when syncing
Schema cache refresh ran more often than needed

rel2022.11.01

Features

Added support for relative date filters in syncs

Fixes

Salesforce schema cache could become stale in some situations
Correctly map qualified types from Athena to Polytomic
BigQuery date filters could fail with a SQL error for DATE fields
Improved performance when creating Netsuite models
Run-after syncs could stop running if the upstream sync completed at the same time as a network outage

rel2022.10.25

Features

Object and Array fields will be created as JSON fields in BigQuery

Fixes

Some Salesforce warnings were erroneously classifed as warnings
CSV export links were unclickable in Outlook desktop in some circumstances
Affinty merge errors were not handled correctly when querying models
Syncs from Salesforce History objects failed due to missing LastModifiedDate
Additional sync execution performance improvements

rel2022.10.19

Features

Models page loads faster for users with many models
Sync execution performance improvements

Fixes

Null values for LastUpdatedAt on Salesforce objects could case sync errors
Syncs to S3 were Queued if another sync was running to S3
Improved query speed for Athena models

rel2022.10.11.03

Execution logs could be prematurely pruned

rel2022.10.11

Features

Query Runner search returns results from all available schemas
CloudSQL connections may use different credentials for network and database authentication

Fixes

BigQuery operations were not retried when an HTTP error was returned
Execution logs no longer require Postgres for storage
Hubspot objects with more than 30 associations could not be synced

rel2022.10.04

Features

Added support for Stripe Prices as a mdoel

rel2022.09.28.01

Support configuration of SQLServer transaction isolation level
Correctly prune execution logs after 7 days

rel2022.09.28

Features

Added support for Hubspot Meetings as a model
Added support for Stripe Subscription Schedules as a model

Fixes

Salesforce queries against large models could fail with "Header Too Large" error
Sync execution logs are now pruned after 7 days
SOQL models using aggregations now support up to 4000 results, working around the Salesforce limitation of 2000
Logging from Athena driver errored when writing entries to Postgres
Athena map values where no value is present for a key could cause an error to occur
Snowflake error counts were incorrect in some situations

rel2022.09.20.01

Fixes

Fixed a bug where more than one organization sharing an SSO domain name caused login to break

rel2022.09.20

Features

Added support for authentication settings in CSV models
Added support for syncing to Amplitude group identities
Added Person URI to Harmonic highlights models

Fixes

Sync configurations involving several models could result in duplicate records in rare circumstances
Sync overrides were erroneously applied cumulatively, which had the potential to cause issues with multiple overrides on the same field
When configuring a webhook sync configuration it was possible to select fields which were not related to previous selections

rel2022.09.13.01

Fixes

Improved handling of long running operations from query runner
Query Athena sources first to improve preview & test sync performance
Record detailed logging from the Athena driver during sync execution

rel2022.09.13

Features

Added support for BCC'ing notifications to administators for on premises deployments
Pagination tokens may be set in subsequent request body for API models
Exposed "Formatted Highlights" as a field for Harmonic people

Fixes

Field write mode was erroneously disabled for Airtable syncs
Work-around for Intercom API issues with IN queries
Specifying the full URL for an API healthcheck path resulted in an error

rel2022.09.06

Features

Added support for setting labeled associations in Hubspot

Fixes

Clicking Refresh Fields in sync configuration refreshes model as well as destination fields
Hubspot record links could be incorrect for users with multiple Hubspot organizations

rel2022.08.30.02

Fixes

Syncs from Azure SQL models with filters applied could error on preview

rel2022.08.30.01

Fixes

Syncs from Athena models with filters applied could error on preview

rel2022.08.30

Features

Added "contains" and "not contains" to Salesforce Target Filters for string fields
Exposed nested Shipbob order information for use in models

Fixes

Query Runner incorrectly formatted date values as timestamps for some integrations
Query Runner omitted columns from SOQL query results if all rows had a null value
Duplicate record counts incorrectly included records excluded based on sync mode
Creating a sync to an existing table with capitalized identifiers could cause an error in some situations
SSH tunnels were left open on system shutdown

rel2022.08.23

Features

Model list may be filtered to only display models created by the user
Syncs to Webhook destinations can configure the request batch size

Fixes

Improved performance when saving a Marketo model
datetime_to_ms function returned result in seconds
Not Empty filter could cause sync to fail in rare circumstances

rel2022.08.16

Features

Added support for syncing to Iterable events
Added "Annual Run Rate (dollars)" to supported Chargebee fields
Added ptctl stats command to the command line interface; this reports on the number of connections and active syncs

Fixes

Sync previews could be missing the identity field values
Values from related models were omitted from SOQL query results
Affinity organizations with duplicate entries in a list could skip updating one of the entries during sync
Using a MongoDB View as a model could result in an error
Allow specification of event ID when syncing to Amplitude
Health checks for Azure blob storage connections did not correctly report errors
Rate-limit headers from Kustomer were not respected in some situations
Replace-mode syncs could result in an empty table if there were no source changes

rel2022.08.10

Features

Added support for Query Runner workbook mode
Added support for creating a model from Query Runner
Added support for setting Google Analytics GA_MEASUREMENT_ID
On-premises deployments may whitelist integrations
Users can override field names when creating new fields during sync configuration

Fixes

Improved sync preview performance
Improved performance when fetching Redshift schema
Organizations could be created in an Affinity list during an update-only sync in some circumstances

rel2022.08.03

Fixes

Building models from Views could result in an error
Pasting a column from the Query Runner schema pasted fully qualified name into the query; only the local name is pasted now
Sync preview performance is improved
Server will fail to start if the Deployment and Key are not set

rel2022.08.02

Features

Added support for authorizing Postgresql connections using client certificates
Added support for connecting to Google CloudSQL Postgresql using an IAM service account
Added support for syncing Marketo Activities and Program Members
Added support for syncing Shipbob Product fulfillment line items

Fixes

~Organizations in the Affinity global namespace were incorrectly identified as "updates"~
Organizations could be added to Affinity lists in an update-only sync
Affinity array values were not formatted using their element types before syncing
Organizations in the Affinity global namespace were included in Affinity model results
Basic auth header could be omitted in paginated requests in some circumstances

rel2022.07.25

Features

Added support for syncing Shopify data
Added history to Query Runner

Fixes

Text values were erroneously removed from sync configurations under some circumstances
Reduced Zendesk overhead when syncing users without organization membership
It was possible to have duplicate field mappings in a sync configuration, which led to ambiguous behavior
Single record syncs to Salesforce could fail under some circumstances
Hubspot association lookup errors could cause sync to fail, rather than reporting as an error
Significantly educed memory consumption for large syncs
Added support for a single operational bucket for all Polytomic data
Netsuite picklist fields were not mapped to their internal identifier during sync
Salesforce numeric fields did not have the full complement of filters exposed when used in a target filter

rel2022.07.19

Fixes

Improved performance of Query Runner SQL editor
Syncs interrupted by system shutdown will be restarted
Extraneous warnings were logged in some situations for syncs to Salesforce
Affinity test record links could result in a "not found" error
Enriching companies from Harmonic would result in long-running syncs in some situations
Alphabetize schema and field list in Query Runner
Test file created when create a sync to blob storage (S3, Azure Blob Store, Google Cloud Storage) was not cleaned up
When creating a model for a SQL connection with no tables available we now default to the View selector

rel2022.07.15.1

Features

Include environment variable to tune the garbage collector
Add ptctl command to modify sync feature flags

rel2022.07.15

Fixes

An extension is no longer required when creating a sync to a new file in S3
SSO configuration can be updated via the command line

rel2022.07.14

Features

Support for writing to Hubspot Line Items and Products

Fixes

BigQuery Projects and Datasets labeled correctly in Query Runner
Query runner will now insert at the cursor position instead of at the end of the query.
Temporary tables for BigQuery are now cleaned up properly if the job fails
Hubspot products and line-items (#2951)
A fix to the power-on test which could cause startup to fail

rel2022.07.13

Fixes

Athena connections could erroneously pass healthcheck
S3 connections now return the underlying errors encountered during connection
Salesforce warning counts could be incorrect
Added support for using an S3 bucket for job payloads

rel2022.07.12

Features

S3, Google Cloud Storage, and Azure can now be used to store job payloads.
Query Runner and Model Editor now support find and replace (use cmd/ctrl + F)

Fixes

Fixed a bug where the record log would silently coerce values.

Behavioral Changes

When multiple models are involved in a sync, the additional models are
left-joined (to the identity model) instead of outer-joined.

rel2022.07.11

Features

Support for the Polytomic Terraform
Provider
Improved performance and ease of use for Query Runner
Added support for globally disabling record logs
Users editing a sync are added to the error notification list

Fixes

Sync preview could fail when creating a sync to a new S3 file
Hubspot syncs could fail when models had many propreties
Running into our internal rate-limiter could cause Hubspot syncs to fail in
some situations
Comparing fields of different but compatible types could cause SQL errors
Error retries may be disabled for individual syncs

rel2022.07.05

Features

Added support to command line interface for creating users from CSV files

Fixes

S3 bucket names are now handled consistently with the application; either
s3://bucket, bucket, or bucket/prefix is now accepted everywhere
An inaccessible Athena catalog prevented any catalogs from being displayed
Retry execution errors button could return spurious error
A performance improvement when creating Marketo models

rel2022.06.30.01

Fixes

Only show authentication methods configured for a deployment
Do not retry errors from previous sync executions if the identity mapping has
changed

rel2022.06.30

Features

Added support for using Stripe Payment Intents as a source
Added support for using Shipbob data as a source
When creating Airtable fields from Polytomic you can override the Airtable
type (Airtable Enterprise only)
Added support for Airtable metadata as a source (Airtable Enterprise only)
Added support for running in ECS Fargate on premises

Fixes

Using an empty Harmonic watchlist as a model resulted in an error
Records with no non-empty fields could cause an error when synced to SFDC
Improved performance of introspecting Athena tables

rel2022.06.28

Features

Added support for syncing to customers in Kustomer
Added support for using customers and companies from Kustomer as models
Added support for using Stripe Accounts as a model
Added support for creating new Airtable tables & columns when creating a sync
Added support for OAuth client credentials flow with HTTP API connections

Fixes

The command line interface has been renamed to ptctl
Improved performance of enumerating Athena schemas and tables
Improved perforamnce of joining Companies & People from Harmonic Saved
Searches and Watchlists
Improved handling of archived contacts when syncing to Intercom
Rate limit headers were not respected by HTTP API models
Match Intercome user emails case insensitively
Support using numeric custom properties in Hubspot for associations
Only one record was updated in Airtable when multiple matched the identity mapping
Table names displayed in Query Runner were uppercased
Null values were included with Salesforce payloads for relationship lookup by
external ID fields, resulting in record errors

Behavioral Changes

Null value semantics were not correct for some sync destinations; by default
Polytomic will not overwrite existing data with null values.
Polytomic will no longer attempt to traverse joins if the joined field value
is empty.

rel2022.06.21.01

Retry button was not displayed in sync history table

rel2022.06.21

Features

Added support for setting a maximum offset when paginating through API results
Added record logging support for Zendesk syncs

Fixes

Malformed error CSV from Salesforce could cause the actual errors to be
supressed
Reduced memory overhead for syncs to Snowflake
Test sync results had the identity field omitted in some situations
Number of warnings recorded for a sync is now capped

rel2022.06.13

Features

Add support for syncing to Harmonic watchlists
Add support for syncing to Google Cloud Storage
Add support for syncing to Azure Blob Storage

Fixes

Only changed records will be synced from models with tracking fields & a
unique identifier set
API models could wait for responses indefinitely
Fixed compatibiltiy with MySQL 5.x
Relationships were unable to be set in some Salesforce syncs
Syncs from Cosmos could report credential error when reading from reading from
regions

rel2022.06.07

Features

Add support for syncing from Survicate
Notifications are sent when a connection becomes healthy
Salesforce object models automatically set their unique identifier (Id)

Fixes

API limits could be set without specifying how to use them during requests
Deleted model fields are removed from syncs
Unsigned ints could fail to coerce types properly
Force resyncing an update only sync would truncate the destination table
Improved performance when creating BigQuery models
Hubspot validation errors could be erroneously reported as "invalid value"
Syncs using Harmonic company enrichment could miss changes that occured in
Harmonic's data set
Users were unable to create models for some Opportunity-related Salesforce
objects

rel2022.06.01

Features

Added support for syncing to S3
Added support for Marketo as a model
Added support for using all Harmonic Saved Searches as a model

Fixes

Null characters in model example values could produce an error when saving
Identity field value was omitted from Preview records in some situations
Syncs to Stripe could incorrectly double total record count

rel2022.05.24

Features

Added record log support to Airtable, Affinity,
Exposed Harmonic updated_at

Fixes

Default Salesforce request limit was not set when creating a new connection in
some situations

rel2022.05.20

Fixes

Connections that were not used by a sync are now actively health-checked
OAuth tokens refreshed in the course of a health check are retained for future
requests
In some situations Polytomic would attempt to refresh a previously expired
token
Creating a new Snowflake schema could result in the display of spurious errors

rel2022.05.19

Features

Added support for custom rate limits on Zendesk connections
Added support for append-only syncs to Google Sheets

Fixes

OAuth tokens could expire before being refreshed in some situations
Updated record log was incomplete for Salesforce destinations in some
situations
Syncs with model filters included the filter field value when de-duplicating
Some model filters failed when used with Postgres UUID fields
Pipedrive rate limit backoff was not respected in some situations
Errors from enumerating BigQuery schemas were not reported
Syncing numeric fields from BigQuery to Google Sheets resulted in values
expressed as fractions

rel2022.05.01.01

Fixes

Fixed speed regression with condition evaluation during syncs
Salesforce syncs may not pull the latest token when a error is encountered

rel2022.05.11

Features

Added support for syncing to Dialpad Contacts
Added support for syncing to BigQuery Record and Repeated fields
API models support transforming the pagination token during requests
Model editor displays syncs which currently use the model

Fixes

Record logs did not include identity value for syncs to Redshift and BigQuery
Applying conditions Postgres UUID fields resulted in an error
Field mapping create-only mode was not respected for some sync configurations

rel2022.05.04

Features

Pipedrive Deals now supports fetching deleted records

Fixes

Improved error localization/messaging around the app
Google sheet models can now survive renaming
Create-only field in syncs now have create-only mode by default
Airtable models can be created even if the table is empty

rel2022.04.27

Features

API models can extract next-page URL from HTTP response headers

Fixes

Handle cases where Hubspot sends unexpected association data
SSH tunnels are now health-checked indepedently of the database connection
Reduced memory usage for most deployments

rel2022.04.21

Fixes

Fix a bug causing unnecessary connections to databases/caches

rel2022.04.19

Features

Added support for custom HTTP headers on Webhook connections

Fixes

Improvements to queueing when multiple syncs attempt to write to the same
destination
Report more helpful errors from Snowflake syncs
Polytomic managed schemas will use uppercase field names for Snowflake
Non-numeric postal codes in Pardot could cause sync errors

rel2022.04.12

Features

SQL connections can now sync into any schema (as well as create new schemas)
Create/Update mode for Redshift
Support setting "Default Organization" on Zendesk Users.
Harmonic Watchlist support

Fixes

Various UI/accessibility improvements with focused fields
Intercom integration no longer double-counts records in sync summary
Various fixes and improvements for Netsuite

rel2022.03.29

Features

Added support for using Pipedrive as a source
User-defined fields may now use JSON path expressions

Fixes

Null values in numeric fields were previously expressed as 0 in some cases;
they are now correctly expressed as null values.

rel2022.03.15

Features

Added Replace mode support to MySQL syncs
Added Chargebee Product Catalog 2.0 support
Added support for offset-based API pagination

Fixes

Tables were not truncated when performing a BigQuery full resync
Concurrent sync executions against a single destination could result in
deadlock

rel2022.03.08

Features

Added support for Netsuite custom list fields
Added support for Affinity custom fields in models

Fixes

Handle additional error cases when writing to Affinity
Quotes in Intercom field values were incorrectly escaped
Improved error reporting for syncs to Redshift

rel2022.03.01

Features

Added support for Netsuite Employee models
Adjusted Snowflake & Redshift destinations for improved warehouse utilization

Fixes

MongoDB ObjectIDs are correctly emitted when used in a sync
Operations that return an error when writing to Snowflake will be retried

rel2022.02.22

Features

Improved performance when syncing to Snowflake

Fixes

Provide more informative error messages for query errors
Syncing from Cosmos could send unnecessary records in some cases
Sample values for Mongo array fields showed a single value, rather than the
array
Harmonic enumerations are coerced to human readable values

rel2022.02.15

Features

Added support for connecting to Netsuite
Added support for duplicate partition key values in Cosmos collections
Exposed Harmonic emails & Highlights as formmatted strings
Added support for enriching company data from Harmonic based on website URL

Fixes

When creating a sync to a newly created table, Preview could fail
Records synced to Intercom could be counted twice in the total

rel2022.02.08

Features

Primary personal & work emails are now exposed for Harmonic persons
Syncs will create values in Airtable Select fields if needed

Fixes

Nullable UUIDs in Postgres databases could deserialize incorrectly in some
situations
Salesforce connections handle expiration of tokens more gracefully
[Datetime expressions](https://docs.polytomic.com/docs/configuring-an-http-api-model#datetime-expressions-in-headers-and-query-parameters are now case insensitive
Retrying sync executions with a large number of errors could fail in some
situations
Cosmos models will now retry requests when rate limited
The initial sync from Cosmos could be incomplete due to partitioning
SQL destinations will truncate the target table when forcing a full resync

rel2022.02.02

Features

Associations between Hubspot custom objects are now supported
Reduced requests needed when syncing Hubspot

Fixes

BigQuery sync failures could go unreported if failure occured after the "load"
jobs
Numeric identity values could cause spurious duplicate warnings
Syncs to Snowflake could report inaccurate field counts
Empty strings could error when synced to an Airtable select field
"Does not contain", "does not end with" filters would evaluate to false when
the value was empty
Numeric values could error when synced to Airtable under some circumstances
Google Sheets models could fail to detect column types

rel2022.01.26

Features

Added support for create+update syncs to Big Query
Airtable Record IDs may be used in sync identity mappings
Traction metrics for Harmonic companies may be added to models
Sync execution errors may be retried for Hubspot syncs

Fixes

Reading a blank padded character fields in Redshift could fail
Values are written to Google Sheets as user-entered values
Sync execution warnings were not exposed to the user in some situations

rel2022.01.18

Features

Add Hubspot Line Items as a model source
Add support for customizing API model fields

Fixes

MySQL JSON field values could be returned as strings rather than objects in
some situations
Correctly sync Airtable duration fields
Hubspot syncs could trigger rate-limiting behavior
Do not attempt to enrich empty company value from Harmonic
Correctly trim Redshift padded character fields
Sync preview could fail if new fields were added to a Polytomic managed
database target
Airtable errors were not linked to specific records in some cases

rel2022.01.11.01

Fixes

Only query Marketo for unique identity values

rel2022.01.11

Features

Add support for syncing to Redshift
Add support for enriching models with Harmonic company data

Fixes

Make Marketo search batch size configurable via MARKETO_BATCH_SIZE
environment variable
An unhealthy connection could not be reconnected via the user interface in
some situations
Retry syncs with non-string identity mappings could fail to complete
Is empty/Is not empty filters for numeric fields could construct invalid
queries
Reduce sync concurrency to accommodate Zendesk job limits

rel2022.01.04.03

Fixes

Marketo auth token could expire while searching for matching Leads

rel2022.01.04.02

Fixes

Avoid "too few results" errors from Marketo

rel2022.01.04.01

(There is no rel2022.01.04 due to a build issue.)

Features

Add support for syncing to Pipedrive
Add support for using Dialpad aggregate reports and webhook call data as
models

Fixes

Job locks could be held longer than necessary in some cases
Improve handling of syncs to Snowflake
Report ignored sync errors (ie, when ignoring invalid references) as rolled up
warnings
Adjust request batching when Marketo reports too many possible results
Improve support for large Zendesk syncs
Improve support for associating Zendesk users & organizations

rel2021.12.23

New Features

Add support for replacing Postgresql tables during sync execution
Improve speed for syncs to Snowflake
A maximum sync duration may be set via the environment

Fixes

Display object values correctly in Query Runner results
"contains" filters for BigQuery models resulted in an error in some situations

rel2021.12.21

New Features

Add support for reading Dialpad call logs
Add support for syncing Airtable attachment fields
Add support for using Airtable formula fields in identity mappings

Fixes

Improved handling of Zendesk associations
Increased performance for syncs to Snowflake destinations
Invalid reference errors from Salesforce were not exposed sometimes
Sync failure notifications could be sent without an error shown
Some read-only fields were excluded from the sync record logs
Clustering keys are set when creating Snowflake tables
Newly created MySQL tables are prefixed with polytomic rather than being
created in a schema

rel2021.12.03

Fixes

Don't fail partially successful BigQuery jobs
More efficient fetching of syncs list
Fix Snowflake batch inserter
Report an error if all records in SQL job fail
Support Snowflake VARIANT/ARRAY columns for create/update
Explicitly report timeouts to user rather than silently failing
Reduce over-fetching of SQL schemas
Correctly handle null arrays coming from Snowflake

rel2021.12.01

New features

Model filters for Cosmos DB

Fixes

More efficient Snowflake querying (async mode)
Fix Snowflake writes for arrays and JSON objects for replace syncs
Allow manual syncs to be selected for sync-after
More stability in the face of errors when writing to SQL databases
Fix 'Last modified' to only display on saved models
Fix memory leak on sync status page
Exclude read-only types from Airtable destinations
Performance improvements for sync history page

rel2021.11.24

New features

Show 'Last modified by' info on models and syncs

Fixes

Surface failures from Smartsheet jobs
Fix duplicate header in record logs
Correctly format dates for Google Sheet exports
Fix bug with connecting to Airtable bases
Improve performance of API model editor
Fix bug with per-field 'create-only' mode when applied to HubSpot destinations
Fix bug with some date formats when writing to Salesforce

rel2021.11.17

Fixes

More robust handling of Affinity source data
More robust writing to SQL databases when some records fail
Correctly handle embedded commas when syncing Google Sheets to Snowflake
Show 'Run after' syncs list on hover on syncs page
Remove 'Browse' button from saved Google Sheets connections
Enforce model refresh if query changes
Robust handling of large (300k+) simultaneous record errors when syncing
Decode Hubspot array fields correctly
More efficient browser memory usage when viewing sync history page
Faster load time in Query Runner for large schemas
Fix occassional 'unreachable fields' display bug in sync config

rel2021.11.10

New features

SQL Runner supports SOQL queries
Add 'Contains' model filter with arrays
Faster performance of SQL runner schema display with many tables
Add body parameter for API source models on POST

Fixes

Handle Hubspot array fields correctly
Improve memory pressure on sync history page
Correctly format dates for snowflake-destination replace syncs
Fix occasional 'unreachable fields' display bug

rel2021.11.04

New Features

Ability to define models from SOQL queries
Allow type overrides on API models
When writing to databases, automatically generate target column when new model field is added to sync
Add support for notifications on sync warnings
Record duplicate identity values as sync warnings

Fixes

More robust handling of nil values in Affinity
More robust handling of Airtable dates
Fix bug where per-field sync mode overrides weren't getting saved
Migrate to new HubSpot OAuth scopes
Remove limit parameter from API fieldsets
More robust handling of empty identity values
User setting toggle would sometimes be out of date
Do not fail sync on missing identity value
API sources now terminates pagination on duplicate results
Support datetime strings where offset is +/-hhmm (#1465)

rel2021.10.26

New Features

New source integration: Databricks
New destination integration: Smartsheet
Log Salesforce inserted and updated records
Log duplicate Salesforce records as warnings encountered during sync
Allow use of readonly Salesforce fields in identity mapping
Support for querying MongoDB
Support for ARRAY and BIGNUMERIC types in BigQuery

Fixes

Sync history record log now downloads as CSV rather then opening in browser as plaintext
Fix error when saving sync with null overrides
Fix connection error when connecting to Freshdesk
Faster querying of Google Sheets with many empty rows

rel2021.10.19

New Features

Enable string target filters for Salesforce
Enable logging to S3 of total records synced
Support for Microsoft 365 SSO

Fixes

Performance improvement for displaying large number of tables in SQL database model editor's table selector
On prem deploys can specify valid HTTP origins
Fail syncs with invalid configurations
Do not fail when a single record cannot be encoded
Performance improvement for large syncs to BigQuery

rel2021.10.13

New Features

CSV connections and models support headers and query params
Expose non-company entities for Harmonic

Fixes

Filter duplicate model mappings from override list
Better performance for dropdown menus
Fix rare case where a sync fails but still displays 'Running'
Better displaying of long strings in banner alerts

rel2021.10.06

New Features

Partial sync failure notifications
Syncing people objects from Harmonic
Support for generating models from SQL views
Expose Chargebee payment source fields in model editor

Fixes

More edge-case stability for Salesforce syncs
Fix parse error for SQL queries that contain URLs with a '?' character
Fix Chargebee edge case type errors
Smoother UI flow for Okta SAML
Fix occasional display of field ID rather than field name in sync preview

rel2021.09.28

New Features

Okta SAML login support
Much faster syncs from Stripe
Show errored record details for HubSpot sync errors
Expose Stripe Invoice and Invoice Line Item objects

Fixes

More thorough exposure of available fields for API models
More stability for HubSpot sync edge cases

rel2021.09.21.01

Fixes

Disable Salesforce composite API when a batch size is set

rel2021.09.21

New Features

New sources: AWS Athena, Azure Cosmos DB, Chargebee, Harmonic
New destination: Snowflake
Allow more than one sync override for the same model field
Faster interactions and saves for many-field models in model editor
When writing to SQL databases, Polytomic's auto-generated schema now includes created_at and updated_at columns

Fixes

Sort sync list by destination object name
Improve saving performance on model config
Optimisations for HubSpot syncs
Include Polytomic client ID when making Salesforce requests

rel2021.09.13

Fixes

Correct set HubSpot associations in more edge cases.
BigQuery schema generation now maps Arrays and Objects to String instead of Bytes.

rel2021.09.10

New Features

New sources: AWS Athena, Chargebee
Add ability to specify identity value in sync preview
Add ability to specify target filters (and filter logic) in addition to existing model filters
More efficient SFDC syncs
Show SQL database types in model editor
Include the execution type with webhook payloads
Add automatic verification for JSON path in API models

Fixes

Correct mapping of HubSpot associations in some cases
More robustness in the face of Google Sheet rate limits
Sort SQL table names in model editor dropdown
Enforce a platform limit for Salesforce calls
Fix mistaken identity values in some HubSpot cases

rel2021.09.01

New Features

New destination: Segment
Ability to add sync filters and overrides for API model fields
Ability to add filter logic for target fields, not just source ones
Ability to use datetime expressions in string constants
Ability to specify a minutes-threshold environment variable LONGSYNC_THRESHOLD for long-running-sync alerts
Add "Only sync if different" option for Salesforce syncs
Field-level create-only mode now will also write data if the record exists in the destination but the corresponding field is empty
Remove default last-24-hours filter on sync history page
Support for connecting to MySQL via SSH tunnel

Fixes

Fix URL to download error log for on-prem
Set a sync as failed if maximum number of retries is exceeded
Allow Marketo jobs to retry indefinitely

rel2021.08.18

New Features

Add ability to specify a constant string value in sync mappings
Add ability to override string values in sync mappings with user-entered value
Add ability to write to MySQL and PostgreSQL with auto-generated schemas

Fixes

History for continuous syncs now only lists syncs with non-zero updates

rel2021.08.11

New Features

New source: Azure Synapse
Sync filter predicates for strings, numbers, and dates
Sync filter logic
Filters for sync history page
Much faster loading of Snowflake tables in model editor
In-app documentation link

Fixes

Fix bug when syncing to Pardot checkboxes
More detailed error reporting for Salesforce edge cases

rel2021.07.27.03

Fixes

"Write null values" setting was not respected for Salesforce syncs in some
situations

rel2021.07.27.01

Fixes

Changed records could be missed in some edge cases

rel2021.07.27

New Features

New destination: MySQL
Sync diffs now only consider the fields in a sync mapping rather than all fields in parent models
New sync schedule option: run after other syncs complete
Expose Hubspot enumerations as target values
Add support for Hubspot Tickets
Add additional types to Salesforce models (picklist, address, int)

Fixes

More generous Snowflake timeout when checking its connection
More generous SQL Runner timeouts for long-running queries
Fix SQL Runner scrollbar bug on Chrome for macOS
Fix SQL Runner empty-row results for BigQuery
Validate unreachable fields and invalid filters on sync save
More efficient connection health checks
Unadulterated display of Byte[] types from BigQuery
Expose Affinity ID for Person, Org identity
Affinity rate-limiting improvements
Fix weekly sync Sunday display bug
Move Google Sheet tab selection to model config
Extend Snowflake timeout when checking a connection
More efficient handling of simultaneous SQL database connections
More efficient handling of HubSpot rate limits
More efficient handling of Pardot rate limits
Normalize case of email addresses
Format empty arrays for Hubspot
Add additional types to salesforce mapping
Syncs page now orders them deterministically
Clearer error messages for API sources

rel2021.07.13

New Features

New-user onboarding flow
New source: CSV URL
New destinations: BigQuery, Klaviyo, and Google Ads
Add field-detail view to filters
Support non-equality filters on Stripe invoices
More efficient Salesforce syncing
More efficient syncing from Stripe

Fixes

More stable Salesforce token refreshes
More accurate detection of Salesforce external-ID errors
Fix some cases where filtering on Salesforce fields was not applied
Use correct encoding for Salesforce null values when mixing Bulk v2 and composite APIs
Correctly handle Salesforce fields without labels
Join target field in model query editor now also displays its model
Show connected user's email for Google Sheets connections
Show sync error count while hovering over partial-error icon
Pardot bugfixes
Sync history duration was inaccurate in some cases
Sanitize API source pagination
Support parameterizing arguments on API sources
Add showFieldDetails to sync filters
Only short-circuit processing w/no filters set
Set IsAssociation for all SFDC relationship ids
Sync summary mappings view now shows field details on hover
Show error message on unreachable sync filter fields

rel2021.06.29

New Features

New destination: Pardot
Display SFDC and Marketo API count even when user has not set a limit
Less SFDC API usage

Fixes

More efficient syncing from Stripe Customer record
Show response status code in webhook request log
Fix cases where error counts would be inaccurate
Fix cases where completed time would not update for SFDC Bulk v1
Store full request URL with webhook logs
Remove duplicate Lead errors if 'Skip converted' is set

rel2021.06.24.01

Fixes

Correctly enable Salesforce integration based on configuration

rel2021.06.24

New Features

Expose calculated fields on Salesforce destinations
Support boolean destination field filters for Salesforce
Hide unreachable fields in sync mappings
Use connection logos for sync summary field logos
Add field API name to sync 'Show details' view
Enable request logs for webhook syncs
Ability to add metadata to webhook syncs
Readable errors for failed HTTP API-source syncs

Fixes

More stability for Stripe joins
Nulls now copied correctly for SFDC Bulk V1 batches
Security improvements
Sync errors now consistently surface during retries
Correctly display nested BigQuery schemas in table picker
Decreased network load per connection
Show local field label in "Inbound joins" UI entry
Hide sync filters if mappings are empty

rel2021.06.08.02

Uploads to Marketo were erroneously marked as failed, regardless of final
status.

rel2021.06.08.01

Log job & failure information from Marketo

rel2021.06.08

New Features

New destination: Freshdesk
Ability to clone syncs
Show timestamp of when a connections objects were last retrieved
Token pagination for REST API sources

Fixes

Correctly infer array and object model fields
Fix instability when unable to retrieve Zendesk custom fields
Platform support for denormalized records (#745)
Add support for Token pagination for APIs

rel2021.06.01

New Features

Ability to sync from arbitrary APIs
Ability to sync to arbitrary webhooks
Ability to trigger syncs through API
Ability to add all fields from a model in sync mapping in one click
Show field types in model query editor
Support for syncing multi-select strings to HubSpot
New 'completed with errors' status in sync history page

Fixes

Increased efficiency of cross-model joins
Increased stability of MongoDB connections
Order models by name in sync mapping dropdown
Better support for Stripe sub-fields in model builder

rel2021.05.25

New Features

Add date filter to sync filters
New destination: Webhooks
New source: HTTP APIs

rel2021.05.18.05

Fixes

Upgrade Snowflake client library
Add support for Big Query NUMERIC fields

rel2021.05.18.04

Fixes

Expose SQL errors which could lead to premature query termination
Add support for setting region on bigquery clients
Reduce memory consumption used for duplication record detection

rel2021.05.18.03

Fixes

Add diagnostic logging for records excluded from syncs based on filters,
change detected, and de-duplication

rel2021.05.18.01

Fixes

Fix situation where a Salesforce error could result in the sync halting

rel2021.05.18

New Features

New destination: Facebook Ad Audiences

Fixes

Fix sporadic instability with Salesforce syncs
Better type handling for Iterable syncs
Improved logging of Salesforce integration
Fix occasional error with HubSpot associations

rel2021.05.12

New Features

New destinations: Iterable and Amplitude
Weekly syncs summary email
Add ability to associate records in HubSpot syncs
Add support for 'does not equal' sync filtera

Fixes

8x speedup for Snowflake queries using multiple tracking fields
Fix issue where scheduled syncs sometimes don't run
More stability for HubSpot syncs

rel2021.05.04.01

Fixes

Fix daily-schedule bug that recorded wrong minute value

rel2021.05.04

New Features

New sources: HubSpot and Stripe
Add null/not null sync filters for Salesforce target fields
Add sync filters for Google Sheets model fields
Show full database schema in SQL Runner
Add create and update-only sync modes for HubSpot
Expose equality sync filters for Hubspot model fields

Fixes

Faster loading of model and sync pages
Format dates properly when syncing to HubSpot
HubSpot test syncs sometimes did not show the correct records
Miscellaneous UI improvements

rel2021.04.27.01

New Features

Tracking fields are now optional
New UI design
SQL runner: run ad-hoc SQL queries on your databases
Ability to set create-only mode per-field
Show/hide field details in sync mapping config

Fixes

Hard timeout for sync preview operations that take too long
Show on-premises execution log URL for any started execution

rel2021.04.21

Fixes

Show on premises log link for any started execution
Fix occassional stability issue with Postgres connections
Better performance for large Airtable tables
Fix bug where a connection would mistakenly save on successfull OAuth
Expose Affinity CRM relationship fields for sync
More stable recovery in the face of errors from Google Sheets's API
Prevent occasional record duplicates in HubSpot syncs
Better support for Airtable formula fields

rel2021.04.13

New Features

Add support for syncing to Intercom companies
Add support for syncing from Airtable Views

Fixes

Sync field mappings are now listed in model-field alphabetical order
Faster fetching of Airtable bases and tables
Improvements for syncing to Affinity CRM
Redirect user to login on session expiration
Improved display of BigQuery table names
More stability when refreshing Google Sheets connections

rel2021.03.30

New Features

Cron sync scheduling via 'Custom' schedule option
Ability to sync to Stripe Customers and Subscriptions
Unhealthy-connection destinations are now marked as such on the syncs list page

Fixes

Security upgrades
Query speed improvements
(on-prem) More efficient database space utilisation

rel2021.03.17.02

Handle additional errors during Salesforce bulk job creation
Reduce the number of SQL queries needed when joining fieldsets

rel2021.03.17.01

Fixes

Optimize SQL queries when multiple tracking fields are used
Improve caching behavior for queries executed during a sync execution

rel2021.03.17

New Features

New destinations: Intercom, Affinity CRM
Syncs page is now grouped by destination
Model page query fieldsets are minimized by default
Expose support for sync field filters on Salesforce fields

Fixes

Improve load speed for syncs and model pages
Lower memory usage during syncs, especially for large ones
More efficient related-object syncing in Salesforce
Do not return model page error if we cannot enumerate SFDC schema
Google Sheets syncs would fail in rare conditions
Add ability to ignore invalid references
Better automatic model validation when executing a sync
Clean-up relationships from removed fields
Do not attempt to sync records we were unable to map to destination
Gracefully handle search errors from Hubspot

rel2021.03.02

New Features

New sync destination: Airtable
Syncs page now groups syncs by destination
Make query name sticky when scrolling through model fields selector
Add join indicators on model page

Fixes

Syncing MongoDB to Google Sheets would sometime fail
Cancelling a Salesforce sync from Polytomic after Salesforce has already written all the data should not count as a complete job
Show error message for the specific case of failing to update a Google Sheet
Expose Salesforce IDs as identity mapping targets
Security improvements

rel2021.02.23

New Features

Ability to set filters on model fields in a sync configuration: is null/not-null and true/false for boolean fields.
Banner confirmation of invitation email when a new user is added

rel2021.02.16.04

Retry network requests in the event of unexpected hang up

rel2021.02.16.03

Username and password were not always passed during the Redis client
construction

rel2021.02.16.02

Errors when searching Marketo could result in a stopped sync

rel2021.02.16.01

Connections to SQL databases were prematurely removed from the connection
pool.

rel2021.02.16

New Features

Expose 'sync all records' setting for all targets, not just Salesforce
New sync source: Airtable
Support for syncing to and from very large Google Sheets
Allow and respect update-only and create-only fields in a create-update sync
Indicate a Salesforce connection as unhealthy if Salesforce has run out of API calls for the day, even if that has nothing to do with Polytomic
Update weekly sync summary email to summarize the whole week

Fixes

Occasional inability to change a connection name
Edge cases that would result in inaccurate updated-record counts in the sync history
Introduce restriction of query relationship targets to existing model fields only
Speed optimisations for large-data syncs
Occasional errors for calculated columns in Google Sheets

rel2021.02.03.02

Update upstream Redis library to incorporate Redis Cluster fixes

rel2021.02.03.01

Fixes

Tracking field names were not always quoted correctly

rel2021.02.03

New Features

Implemented a weekly sync report

Fixes

Snowflake field labels are now title-cased rather than uppercase
Accurate indication of whether Salesforce relationship target fields are creatable or updateable
Fix some cases where cancelling a running sync would result in nothing
Better handling of default Salesforce batch sizes
More efficient Salesforce upserts in bulk mode

rel2021.01.26.02

More aggressively cache Marketo target definitions

rel2021.01.26.01

Fixes

Marketo syncs were not using the maximum available batch size

rel2021.01.26

New Features

Clickable download links for sync logs in on-premise deployments
Allow setting syncs to always be full syncs
Show Salesforce account user in its connection configuration

Fixes

Speed optimisations when gathering data for a sync
More efficient Salesforce API usage for some sync types
Further reduction in logging volume
Treat automatically-skipped syncs as failures
Treat Salesforce identity keys as case insensitive

rel2021.01.19.02

Remove logging statements without level filter

rel2021.01.19.01

Fixes

Updating Salesforce related objects using an external ID would sometimes fail
when serial concurrency or fixed batch size was set.

rel2021.01.19

New Features

Add support for update-only or create-only Marketo syncs

Fixes

Creating a new Google Sheets connection would sometimes fail
Security upgrades

rel2021.01.17

Fixes

Marketo syncs could fail to start due to overly aggressive locking.
A sync could fail the first time it was run after changing referenced tracking
fields in the model.

rel2021.01.12.3

Fixes

Fixed regression in model query editing

rel2021.01.12.2

Fixes

Relationships to related Salesforce objects were not set in some
circumstances.
Preview was unavailable when a sync included related records.
Lookup fields on related records were erroneously flagged as required.

rel2021.01.12.1

Fixes

Corrects a bug which could cause duplicate records to be created in
Marketo.

rel2021.01.12

New Features

Add ability to sync to any SFDC object's related-object's ID field
Add AzureSQL support

Fixes

Don't disappear 'full resync' button during sync
Allow the same model field to sync to two different target fields
Remove character limit on query titles
Stability improvements for particular Redis environments
Skip data fields for a particular sync if source system is unavailable
Editing query fields propagates those changes to
relationships using those fields
Fix error preventing logging in to Polytomic Front widget in Front's desktop app
Security updates
Logging improvements

rel2021.01.05

New Features

Add support for Azure SQL connections

Fixes

Namespace all Redis keys for cluster mode compatibility; this includes the
contents of all rel2020.12.23 hot fixes.
Update Docker base image to use Ubuntu 20.04.
Removing a fieldset involved in a relationship could result in the Model
refusing to load.
An error during OAuth connection could result in a half-configured connection.
Hubspot syncs could drop the final record from a batch.

rel2020.12.23.4

Fixes:

Route redis scripts to the correct cluster node.

rel2020.12.23.3

Fixes:

Reference frontend scripts using a domain relative URL

rel2020.12.23.2

Fixes:

Set the Redis namespace for registered task handlers for Redis Cluster
compatibility

rel2020.12.23.1

Fixes

Scope event stream keys correctly for Redis Cluster compatibility

rel2020.12.23

New Features

Add support for Amplitude model
Ability to scroll left and right sides of model edit view independently

Fixes

Display logout button in application sidebar
Edge-case stability for Zendesk syncs
More generous timeout for testing new DB connections
Empty Google Sheet results could fail the sync
A deleted identity field could cause the sync list fetch to silently fail
Allow adding a previously-removed user to an organization
Remove extraneous "Google Sheets" default name prefix from Google Sheet connections
Accept full domains or URLs for Zendesk connection domain
More stable handling for query tests that return an error

rel2020.12.15

New Features

Add support for syncing to Google Sheets
Add support for syncing to Zendesk
Add option to force a fresh authentication for existing OAuth connections
Expose Salesforce external ID fields for sync identity mapping

Fixes

Only display writeable target fields in mapping drop-down
Handle widget view empty state
Refreshing a database table schema in a query after editing field properties would revert said properties
Deleting fields used in an existing sync could prevent new syncs from being created
Deleting errored SSH-tunnel database connections would fail

rel2020.12.09

New Features

Added support for 'views': the ability to display a live subset of the Universal Model based on context (support ticket or customer email) in widgets that can be embedded in third-party customer systems like Front, Salesforce, and Zendesk. Front support only for now.
Added support for a substring-of identity mapping function in addition to the default equality one.

Fixes

Salesforce syncing would fail for certain field mapping type combinations
Under certain conditions, test syncs resulted in an inaccurate record log.
Editing a connection name sometimes resulted in the old name persisting.

rel2020.11.25

New Features

Syncing an array of values or delimited string to a multi-picklist
field in Salesforce will set the picklist values.

When syncing a multi-picklist field in Salesforce, comma,
semi-colon, and null delimited strings will select picklist values,
if a match exists. Syncing arrays to a multi-picklist field will
alow select the picklist values.
Sync filters may be applied to Update syncs.
Updated BigQuery integration to support schema browser.

Previously only SQL queries were supported for BigQuery fieldsets.

Fixes

Deleting a fieldset that is referenced by a sync caused the sync to
disappear from the list.
When running on premises, duplicate root users could be created at
startup.