Changelog

rel2021.12.03

Fixes

  • Don't fail partially successful BigQuery jobs

  • More efficient fetching of syncs list

  • Fix Snowflake batch inserter

  • Report an error if all records in SQL job fail

  • Support Snowflake VARIANT/ARRAY columns for create/update

  • Explicitly report timeouts to user rather than silently failing

  • Reduce over-fetching of SQL schemas

  • Correctly handle null arrays coming from Snowflake

rel2021.12.01

New features

  • Model filters for Cosmos DB

Fixes

  • More efficient Snowflake querying (async mode)

  • Fix Snowflake writes for arrays and JSON objects for replace syncs

  • Allow manual syncs to be selected for sync-after

  • More stability in the face of errors when writing to SQL databases

  • Fix 'Last modified' to only display on saved models

  • Fix memory leak on sync status page

  • Exclude read-only types from Airtable destinations

  • Performance improvements for sync history page

rel2021.11.24

New features

  • Show 'Last modified by' info on models and syncs

Fixes

  • Surface failures from Smartsheet jobs

  • Fix duplicate header in record logs

  • Correctly format dates for Google Sheet exports

  • Fix bug with connecting to Airtable bases

  • Improve performance of API model editor

  • Fix bug with per-field 'create-only' mode when applied to HubSpot destinations

  • Fix bug with some date formats when writing to Salesforce

rel2021.11.17

Fixes

  • More robust handling of Affinity source data

  • More robust writing to SQL databases when some records fail

  • Correctly handle embedded commas when syncing Google Sheets to Snowflake

  • Show 'Run after' syncs list on hover on syncs page

  • Remove 'Browse' button from saved Google Sheets connections

  • Enforce model refresh if query changes

  • Robust handling of large (300k+) simultaneous record errors when syncing

  • Decode Hubspot array fields correctly

  • More efficient browser memory usage when viewing sync history page

  • Faster load time in Query Runner for large schemas

  • Fix occassional 'unreachable fields' display bug in sync config

rel2021.11.10

New features

  • SQL Runner supports SOQL queries

  • Add 'Contains' model filter with arrays

  • Faster performance of SQL runner schema display with many tables

  • Add body parameter for API source models on POST

Fixes

  • Handle Hubspot array fields correctly

  • Improve memory pressure on sync history page

  • Correctly format dates for snowflake-destination replace syncs

  • Fix occasional 'unreachable fields' display bug

rel2021.11.04

New Features

  • Ability to define models from SOQL queries

  • Allow type overrides on API models

  • When writing to databases, automatically generate target column when new model field is added to sync

  • Add support for notifications on sync warnings

  • Record duplicate identity values as sync warnings

Fixes

  • More robust handling of nil values in Affinity

  • More robust handling of Airtable dates

  • Fix bug where per-field sync mode overrides weren't getting saved

  • Migrate to new HubSpot OAuth scopes

  • Remove limit parameter from API fieldsets

  • More robust handling of empty identity values

  • User setting toggle would sometimes be out of date

  • Do not fail sync on missing identity value

  • API sources now terminates pagination on duplicate results

  • Support datetime strings where offset is +/-hhmm (#1465)

rel2021.10.26

New Features

  • New source integration: Databricks

  • New destination integration: Smartsheet

  • Log Salesforce inserted and updated records

  • Log duplicate Salesforce records as warnings encountered during sync

  • Allow use of readonly Salesforce fields in identity mapping

  • Support for querying MongoDB

  • Support for ARRAY and BIGNUMERIC types in BigQuery

Fixes

  • Sync history record log now downloads as CSV rather then opening in browser as plaintext

  • Fix error when saving sync with null overrides

  • Fix connection error when connecting to Freshdesk

  • Faster querying of Google Sheets with many empty rows

rel2021.10.19

New Features

  • Enable string target filters for Salesforce

  • Enable logging to S3 of total records synced

  • Support for Microsoft 365 SSO

Fixes

  • Performance improvement for displaying large number of tables in SQL database model editor's table selector

  • On prem deploys can specify valid HTTP origins

  • Fail syncs with invalid configurations

  • Do not fail when a single record cannot be encoded

  • Performance improvement for large syncs to BigQuery

rel2021.10.13

New Features

  • CSV connections and models support headers and query params

  • Expose non-company entities for Harmonic

Fixes

  • Filter duplicate model mappings from override list

  • Better performance for dropdown menus

  • Fix rare case where a sync fails but still displays 'Running'

  • Better displaying of long strings in banner alerts

rel2021.10.06

New Features

  • Partial sync failure notifications

  • Syncing people objects from Harmonic

  • Support for generating models from SQL views

  • Expose Chargebee payment source fields in model editor

Fixes

  • More edge-case stability for Salesforce syncs

  • Fix parse error for SQL queries that contain URLs with a '?' character

  • Fix Chargebee edge case type errors

  • Smoother UI flow for Okta SAML

  • Fix occasional display of field ID rather than field name in sync preview

rel2021.09.28

New Features

  • Okta SAML login support

  • Much faster syncs from Stripe

  • Show errored record details for HubSpot sync errors

  • Expose Stripe Invoice and Invoice Line Item objects

Fixes

  • More thorough exposure of available fields for API models

  • More stability for HubSpot sync edge cases

rel2021.09.21.01

Fixes

  • Disable Salesforce composite API when a batch size is set

rel2021.09.21

New Features

  • New sources: AWS Athena, Azure Cosmos DB, Chargebee, Harmonic

  • New destination: Snowflake

  • Allow more than one sync override for the same model field

  • Faster interactions and saves for many-field models in model editor

  • When writing to SQL databases, Polytomic's auto-generated schema now includes created_at and updated_at columns

Fixes

  • Sort sync list by destination object name

  • Improve saving performance on model config

  • Optimisations for HubSpot syncs

  • Include Polytomic client ID when making Salesforce requests

rel2021.09.13

Fixes

  • Correct set HubSpot associations in more edge cases.

  • BigQuery schema generation now maps Arrays and Objects to String instead of Bytes.

rel2021.09.10

New Features

  • New sources: AWS Athena, Chargebee

  • Add ability to specify identity value in sync preview

  • Add ability to specify target filters (and filter logic) in addition to existing model filters

  • More efficient SFDC syncs

  • Show SQL database types in model editor

  • Include the execution type with webhook payloads

  • Add automatic verification for JSON path in API models

Fixes

  • Correct mapping of HubSpot associations in some cases

  • More robustness in the face of Google Sheet rate limits

  • Sort SQL table names in model editor dropdown

  • Enforce a platform limit for Salesforce calls

  • Fix mistaken identity values in some HubSpot cases

rel2021.09.01

New Features

  • New destination: Segment

  • Ability to add sync filters and overrides for API model fields

  • Ability to add filter logic for target fields, not just source ones

  • Ability to use datetime expressions in string constants

  • Ability to specify a minutes-threshold environment variable LONGSYNC_THRESHOLD for long-running-sync alerts

  • Add "Only sync if different" option for Salesforce syncs

  • Field-level create-only mode now will also write data if the record exists in the destination but the corresponding field is empty

  • Remove default last-24-hours filter on sync history page

  • Support for connecting to MySQL via SSH tunnel

Fixes

  • Fix URL to download error log for on-prem

  • Set a sync as failed if maximum number of retries is exceeded

  • Allow Marketo jobs to retry indefinitely

rel2021.08.18

New Features

  • Add ability to specify a constant string value in sync mappings

  • Add ability to override string values in sync mappings with user-entered value

  • Add ability to write to MySQL and PostgreSQL with auto-generated schemas

Fixes

  • History for continuous syncs now only lists syncs with non-zero updates

rel2021.08.11

New Features

  • New source: Azure Synapse

  • Sync filter predicates for strings, numbers, and dates

  • Sync filter logic

  • Filters for sync history page

  • Much faster loading of Snowflake tables in model editor

  • In-app documentation link

Fixes

  • Fix bug when syncing to Pardot checkboxes

  • More detailed error reporting for Salesforce edge cases

rel2021.07.27.03

Fixes

  • "Write null values" setting was not respected for Salesforce syncs in some
    situations

rel2021.07.27.01

Fixes

  • Changed records could be missed in some edge cases

rel2021.07.27

New Features

  • New destination: MySQL

  • Sync diffs now only consider the fields in a sync mapping rather than all fields in parent models

  • New sync schedule option: run after other syncs complete

  • Expose Hubspot enumerations as target values

  • Add support for Hubspot Tickets

  • Add additional types to Salesforce models (picklist, address, int)

Fixes

  • More generous Snowflake timeout when checking its connection

  • More generous SQL Runner timeouts for long-running queries

  • Fix SQL Runner scrollbar bug on Chrome for macOS

  • Fix SQL Runner empty-row results for BigQuery

  • Validate unreachable fields and invalid filters on sync save

  • More efficient connection health checks

  • Unadulterated display of Byte[] types from BigQuery

  • Expose Affinity ID for Person, Org identity

  • Affinity rate-limiting improvements

  • Fix weekly sync Sunday display bug

  • Move Google Sheet tab selection to model config

  • Extend Snowflake timeout when checking a connection

  • More efficient handling of simultaneous SQL database connections

  • More efficient handling of HubSpot rate limits

  • More efficient handling of Pardot rate limits

  • Normalize case of email addresses

  • Format empty arrays for Hubspot

  • Add additional types to salesforce mapping

  • Syncs page now orders them deterministically

  • Clearer error messages for API sources

rel2021.07.13

New Features

  • New-user onboarding flow

  • New source: CSV URL

  • New destinations: BigQuery, Klaviyo, and Google Ads

  • Add field-detail view to filters

  • Support non-equality filters on Stripe invoices

  • More efficient Salesforce syncing

  • More efficient syncing from Stripe

Fixes

  • More stable Salesforce token refreshes

  • More accurate detection of Salesforce external-ID errors

  • Fix some cases where filtering on Salesforce fields was not applied

  • Use correct encoding for Salesforce null values when mixing Bulk v2 and composite APIs

  • Correctly handle Salesforce fields without labels

  • Join target field in model query editor now also displays its model

  • Show connected user's email for Google Sheets connections

  • Show sync error count while hovering over partial-error icon

  • Pardot bugfixes

  • Sync history duration was inaccurate in some cases

  • Sanitize API source pagination

  • Support parameterizing arguments on API sources

  • Add showFieldDetails to sync filters

  • Only short-circuit processing w/no filters set

  • Set IsAssociation for all SFDC relationship ids

  • Sync summary mappings view now shows field details on hover

  • Show error message on unreachable sync filter fields

rel2021.06.29

New Features

  • New destination: Pardot

  • Display SFDC and Marketo API count even when user has not set a limit

  • Less SFDC API usage

Fixes

  • More efficient syncing from Stripe Customer record

  • Show response status code in webhook request log

  • Fix cases where error counts would be inaccurate

  • Fix cases where completed time would not update for SFDC Bulk v1

  • Store full request URL with webhook logs

  • Remove duplicate Lead errors if 'Skip converted' is set

rel2021.06.24.01

Fixes

  • Correctly enable Salesforce integration based on configuration

rel2021.06.24

New Features

  • Expose calculated fields on Salesforce destinations

  • Support boolean destination field filters for Salesforce

  • Hide unreachable fields in sync mappings

  • Use connection logos for sync summary field logos

  • Add field API name to sync 'Show details' view

  • Enable request logs for webhook syncs

  • Ability to add metadata to webhook syncs

  • Readable errors for failed HTTP API-source syncs

Fixes

  • More stability for Stripe joins

  • Nulls now copied correctly for SFDC Bulk V1 batches

  • Security improvements

  • Sync errors now consistently surface during retries

  • Correctly display nested BigQuery schemas in table picker

  • Decreased network load per connection

  • Show local field label in "Inbound joins" UI entry

  • Hide sync filters if mappings are empty

rel2021.06.08.02

  • Uploads to Marketo were erroneously marked as failed, regardless of final
    status.

rel2021.06.08.01

  • Log job & failure information from Marketo

rel2021.06.08

New Features

  • New destination: Freshdesk

  • Ability to clone syncs

  • Show timestamp of when a connections objects were last retrieved

  • Token pagination for REST API sources

Fixes

  • Correctly infer array and object model fields

  • Fix instability when unable to retrieve Zendesk custom fields

  • Platform support for denormalized records (#745)

  • Add support for Token pagination for APIs

rel2021.06.01

New Features

  • Ability to sync from arbitrary APIs

  • Ability to sync to arbitrary webhooks

  • Ability to trigger syncs through API

  • Ability to add all fields from a model in sync mapping in one click

  • Show field types in model query editor

  • Support for syncing multi-select strings to HubSpot

  • New 'completed with errors' status in sync history page

Fixes

  • Increased efficiency of cross-model joins

  • Increased stability of MongoDB connections

  • Order models by name in sync mapping dropdown

  • Better support for Stripe sub-fields in model builder

rel2021.05.25

New Features

  • Add date filter to sync filters

  • New destination: Webhooks

  • New source: HTTP APIs

rel2021.05.18.05

Fixes

  • Upgrade Snowflake client library
  • Add support for Big Query NUMERIC fields

rel2021.05.18.04

Fixes

  • Expose SQL errors which could lead to premature query termination
  • Add support for setting region on bigquery clients
  • Reduce memory consumption used for duplication record detection

rel2021.05.18.03

Fixes

  • Add diagnostic logging for records excluded from syncs based on filters,
    change detected, and de-duplication

rel2021.05.18.01

Fixes

  • Fix situation where a Salesforce error could result in the sync halting

rel2021.05.18

New Features

  • New destination: Facebook Ad Audiences

Fixes

  • Fix sporadic instability with Salesforce syncs

  • Better type handling for Iterable syncs

  • Improved logging of Salesforce integration

  • Fix occasional error with HubSpot associations

rel2021.05.12

New Features

  • New destinations: Iterable and Amplitude

  • Weekly syncs summary email

  • Add ability to associate records in HubSpot syncs

  • Add support for 'does not equal' sync filtera

Fixes

  • 8x speedup for Snowflake queries using multiple tracking fields

  • Fix issue where scheduled syncs sometimes don't run

  • More stability for HubSpot syncs

rel2021.05.04.01

Fixes

  • Fix daily-schedule bug that recorded wrong minute value

rel2021.05.04

New Features

  • New sources: HubSpot and Stripe

  • Add null/not null sync filters for Salesforce target fields

  • Add sync filters for Google Sheets model fields

  • Show full database schema in SQL Runner

  • Add create and update-only sync modes for HubSpot

  • Expose equality sync filters for Hubspot model fields

Fixes

  • Faster loading of model and sync pages

  • Format dates properly when syncing to HubSpot

  • HubSpot test syncs sometimes did not show the correct records

  • Miscellaneous UI improvements

rel2021.04.27.01

New Features

  • Tracking fields are now optional

  • New UI design

  • SQL runner: run ad-hoc SQL queries on your databases

  • Ability to set create-only mode per-field

  • Show/hide field details in sync mapping config

Fixes

  • Hard timeout for sync preview operations that take too long

  • Show on-premises execution log URL for any started execution

rel2021.04.21

Fixes

  • Show on premises log link for any started execution

  • Fix occassional stability issue with Postgres connections

  • Better performance for large Airtable tables

  • Fix bug where a connection would mistakenly save on successfull OAuth

  • Expose Affinity CRM relationship fields for sync

  • More stable recovery in the face of errors from Google Sheets's API

  • Prevent occasional record duplicates in HubSpot syncs

  • Better support for Airtable formula fields

rel2021.04.13

New Features

  • Add support for syncing to Intercom companies

  • Add support for syncing from Airtable Views

Fixes

  • Sync field mappings are now listed in model-field alphabetical order

  • Faster fetching of Airtable bases and tables

  • Improvements for syncing to Affinity CRM

  • Redirect user to login on session expiration

  • Improved display of BigQuery table names

  • More stability when refreshing Google Sheets connections

rel2021.03.30

New Features

  • Cron sync scheduling via 'Custom' schedule option

  • Ability to sync to Stripe Customers and Subscriptions

  • Unhealthy-connection destinations are now marked as such on the syncs list page

Fixes

  • Security upgrades

  • Query speed improvements

  • (on-prem) More efficient database space utilisation

rel2021.03.17.02

  • Handle additional errors during Salesforce bulk job creation

  • Reduce the number of SQL queries needed when joining fieldsets

rel2021.03.17.01

Fixes

  • Optimize SQL queries when multiple tracking fields are used

  • Improve caching behavior for queries executed during a sync execution

rel2021.03.17

New Features

  • New destinations: Intercom, Affinity CRM

  • Syncs page is now grouped by destination

  • Model page query fieldsets are minimized by default

  • Expose support for sync field filters on Salesforce fields

Fixes

  • Improve load speed for syncs and model pages

  • Lower memory usage during syncs, especially for large ones

  • More efficient related-object syncing in Salesforce

  • Do not return model page error if we cannot enumerate SFDC schema

  • Google Sheets syncs would fail in rare conditions

  • Add ability to ignore invalid references

  • Better automatic model validation when executing a sync

  • Clean-up relationships from removed fields

  • Do not attempt to sync records we were unable to map to destination

  • Gracefully handle search errors from Hubspot

rel2021.03.02

New Features

  • New sync destination: Airtable

  • Syncs page now groups syncs by destination

  • Make query name sticky when scrolling through model fields selector

  • Add join indicators on model page

Fixes

  • Syncing MongoDB to Google Sheets would sometime fail

  • Cancelling a Salesforce sync from Polytomic after Salesforce has already written all the data should not count as a complete job

  • Show error message for the specific case of failing to update a Google Sheet

  • Expose Salesforce IDs as identity mapping targets

  • Security improvements

rel2021.02.23

New Features

  • Ability to set filters on model fields in a sync configuration: is null/not-null and true/false for boolean fields.

  • Banner confirmation of invitation email when a new user is added

rel2021.02.16.04

  • Retry network requests in the event of unexpected hang up

rel2021.02.16.03

  • Username and password were not always passed during the Redis client
    construction

rel2021.02.16.02

  • Errors when searching Marketo could result in a stopped sync

rel2021.02.16.01

  • Connections to SQL databases were prematurely removed from the connection
    pool.

rel2021.02.16

New Features

  • Expose 'sync all records' setting for all targets, not just Salesforce

  • New sync source: Airtable

  • Support for syncing to and from very large Google Sheets

  • Allow and respect update-only and create-only fields in a create-update sync

  • Indicate a Salesforce connection as unhealthy if Salesforce has run out of API calls for the day, even if that has nothing to do with Polytomic

  • Update weekly sync summary email to summarize the whole week

Fixes

  • Occasional inability to change a connection name

  • Edge cases that would result in inaccurate updated-record counts in the sync history

  • Introduce restriction of query relationship targets to existing model fields only

  • Speed optimisations for large-data syncs

  • Occasional errors for calculated columns in Google Sheets

rel2021.02.03.02

  • Update upstream Redis library to incorporate Redis Cluster fixes

rel2021.02.03.01

Fixes

  • Tracking field names were not always quoted correctly

rel2021.02.03

New Features

  • Implemented a weekly sync report

Fixes

  • Snowflake field labels are now title-cased rather than uppercase

  • Accurate indication of whether Salesforce relationship target fields are creatable or updateable

  • Fix some cases where cancelling a running sync would result in nothing

  • Better handling of default Salesforce batch sizes

  • More efficient Salesforce upserts in bulk mode

rel2021.01.26.02

  • More aggressively cache Marketo target definitions

rel2021.01.26.01

Fixes

  • Marketo syncs were not using the maximum available batch size

rel2021.01.26

New Features

  • Clickable download links for sync logs in on-premise deployments

  • Allow setting syncs to always be full syncs

  • Show Salesforce account user in its connection configuration

Fixes

  • Speed optimisations when gathering data for a sync

  • More efficient Salesforce API usage for some sync types

  • Further reduction in logging volume

  • Treat automatically-skipped syncs as failures

  • Treat Salesforce identity keys as case insensitive

rel2021.01.19.02

  • Remove logging statements without level filter

rel2021.01.19.01

Fixes

  • Updating Salesforce related objects using an external ID would sometimes fail
    when serial concurrency or fixed batch size was set.

rel2021.01.19

New Features

  • Add support for update-only or create-only Marketo syncs

Fixes

  • Creating a new Google Sheets connection would sometimes fail

  • Security upgrades

rel2021.01.17

Fixes

  • Marketo syncs could fail to start due to overly aggressive locking.

  • A sync could fail the first time it was run after changing referenced tracking
    fields in the model.

rel2021.01.12.3

Fixes

  • Fixed regression in model query editing

rel2021.01.12.2

Fixes

  • Relationships to related Salesforce objects were not set in some
    circumstances.

  • Preview was unavailable when a sync included related records.

  • Lookup fields on related records were erroneously flagged as required.

rel2021.01.12.1

Fixes

  • Corrects a bug which could cause duplicate records to be created in
    Marketo.

rel2021.01.12

New Features

  • Add ability to sync to any SFDC object's related-object's ID field

  • Add AzureSQL support

Fixes

  • Don't disappear 'full resync' button during sync

  • Allow the same model field to sync to two different target fields

  • Remove character limit on query titles

  • Stability improvements for particular Redis environments

  • Skip data fields for a particular sync if source system is unavailable

  • Editing query fields propagates those changes to
    relationships using those fields

  • Fix error preventing logging in to Polytomic Front widget in Front's desktop app

  • Security updates

  • Logging improvements

rel2021.01.05

New Features

  • Add support for Azure SQL connections

Fixes

  • Namespace all Redis keys for cluster mode compatibility; this includes the
    contents of all rel2020.12.23 hot fixes.

  • Update Docker base image to use Ubuntu 20.04.

  • Removing a fieldset involved in a relationship could result in the Model
    refusing to load.

  • An error during OAuth connection could result in a half-configured connection.

  • Hubspot syncs could drop the final record from a batch.

rel2020.12.23.4

Fixes:

  • Route redis scripts to the correct cluster node.

rel2020.12.23.3

Fixes:

  • Reference frontend scripts using a domain relative URL

rel2020.12.23.2

Fixes:

  • Set the Redis namespace for registered task handlers for Redis Cluster
    compatibility

rel2020.12.23.1

Fixes

  • Scope event stream keys correctly for Redis Cluster compatibility

rel2020.12.23

New Features

  • Add support for Amplitude model

  • Ability to scroll left and right sides of model edit view independently

Fixes

  • Display logout button in application sidebar

  • Edge-case stability for Zendesk syncs

  • More generous timeout for testing new DB connections

  • Empty Google Sheet results could fail the sync

  • A deleted identity field could cause the sync list fetch to silently fail

  • Allow adding a previously-removed user to an organization

  • Remove extraneous "Google Sheets" default name prefix from Google Sheet connections

  • Accept full domains or URLs for Zendesk connection domain

  • More stable handling for query tests that return an error

rel2020.12.15

New Features

  • Add support for syncing to Google Sheets

  • Add support for syncing to Zendesk

  • Add option to force a fresh authentication for existing OAuth connections

  • Expose Salesforce external ID fields for sync identity mapping

Fixes

  • Only display writeable target fields in mapping drop-down

  • Handle widget view empty state

  • Refreshing a database table schema in a query after editing field properties would revert said properties

  • Deleting fields used in an existing sync could prevent new syncs from being created

  • Deleting errored SSH-tunnel database connections would fail

rel2020.12.09

New Features

  • Added support for 'views': the ability to display a live subset of the Universal Model based on context (support ticket or customer email) in widgets that can be embedded in third-party customer systems like Front, Salesforce, and Zendesk. Front support only for now.

  • Added support for a substring-of identity mapping function in addition to the default equality one.

Fixes

  • Salesforce syncing would fail for certain field mapping type combinations

  • Under certain conditions, test syncs resulted in an inaccurate record log.

  • Editing a connection name sometimes resulted in the old name persisting.

rel2020.11.25

New Features

  • Syncing an array of values or delimited string to a multi-picklist
    field in Salesforce will set the picklist values.

    When syncing a multi-picklist field in Salesforce, comma,
    semi-colon, and null delimited strings will select picklist values,
    if a match exists. Syncing arrays to a multi-picklist field will
    alow select the picklist values.

  • Sync filters may be applied to Update syncs.

  • Updated BigQuery integration to support schema browser.

    Previously only SQL queries were supported for BigQuery fieldsets.

Fixes

  • Deleting a fieldset that is referenced by a sync caused the sync to
    disappear from the list.

  • When running on premises, duplicate root users could be created at
    startup.