Changelog

rel2022.10.04

Features

  • Added support for Stripe Prices as a mdoel

rel2022.09.28.01

  • Support configuration of SQLServer transaction isolation level
  • Correctly prune execution logs after 7 days

rel2022.09.28

Features

  • Added support for Hubspot Meetings as a model
  • Added support for Stripe Subscription Schedules as a model

Fixes

  • Salesforce queries against large models could fail with "Header Too Large" error
  • Sync execution logs are now pruned after 7 days
  • SOQL models using aggregations now support up to 4000 results, working around the Salesforce limitation of 2000
  • Logging from Athena driver errored when writing entries to Postgres
  • Athena map values where no value is present for a key could cause an error to occur
  • Snowflake error counts were incorrect in some situations

rel2022.09.20.01

Fixes

  • Fixed a bug where more than one organization sharing an SSO domain name caused login to break

rel2022.09.20

Features

  • Added support for authentication settings in CSV models
  • Added support for syncing to Amplitude group identities
  • Added Person URI to Harmonic highlights models

Fixes

  • Sync configurations involving several models could result in duplicate records in rare circumstances
  • Sync overrides were erroneously applied cumulatively, which had the potential to cause issues with multiple overrides on the same field
  • When configuring a webhook sync configuration it was possible to select fields which were not related to previous selections

rel2022.09.13.01

Fixes

  • Improved handling of long running operations from query runner
  • Query Athena sources first to improve preview & test sync performance
  • Record detailed logging from the Athena driver during sync execution

rel2022.09.13

Features

  • Added support for BCC'ing notifications to administators for on premises deployments
  • Pagination tokens may be set in subsequent request body for API models
  • Exposed "Formatted Highlights" as a field for Harmonic people

Fixes

  • Field write mode was erroneously disabled for Airtable syncs
  • Work-around for Intercom API issues with IN queries
  • Specifying the full URL for an API healthcheck path resulted in an error

rel2022.09.06

Features

  • Added support for setting labeled associations in Hubspot

Fixes

  • Clicking Refresh Fields in sync configuration refreshes model as well as destination fields
  • Hubspot record links could be incorrect for users with multiple Hubspot organizations

rel2022.08.30.02

Fixes

  • Syncs from Azure SQL models with filters applied could error on preview

rel2022.08.30.01

Fixes

  • Syncs from Athena models with filters applied could error on preview

rel2022.08.30

Features

  • Added "contains" and "not contains" to Salesforce Target Filters for string fields
  • Exposed nested Shipbob order information for use in models

Fixes

  • Query Runner incorrectly formatted date values as timestamps for some integrations
  • Query Runner omitted columns from SOQL query results if all rows had a null value
  • Duplicate record counts incorrectly included records excluded based on sync mode
  • Creating a sync to an existing table with capitalized identifiers could cause an error in some situations
  • SSH tunnels were left open on system shutdown

rel2022.08.23

Features

  • Model list may be filtered to only display models created by the user
  • Syncs to Webhook destinations can configure the request batch size

Fixes

  • Improved performance when saving a Marketo model
  • datetime_to_ms function returned result in seconds
  • Not Empty filter could cause sync to fail in rare circumstances

rel2022.08.16

Features

  • Added support for syncing to Iterable events
  • Added "Annual Run Rate (dollars)" to supported Chargebee fields
  • Added ptctl stats command to the command line interface; this reports on the number of connections and active syncs

Fixes

  • Sync previews could be missing the identity field values
  • Values from related models were omitted from SOQL query results
  • Affinity organizations with duplicate entries in a list could skip updating one of the entries during sync
  • Using a MongoDB View as a model could result in an error
  • Allow specification of event ID when syncing to Amplitude
  • Health checks for Azure blob storage connections did not correctly report errors
  • Rate-limit headers from Kustomer were not respected in some situations
  • Replace-mode syncs could result in an empty table if there were no source changes

rel2022.08.10

Features

  • Added support for Query Runner workbook mode
  • Added support for creating a model from Query Runner
  • Added support for setting Google Analytics GA_MEASUREMENT_ID
  • On-premises deployments may whitelist integrations
  • Users can override field names when creating new fields during sync configuration

Fixes

  • Improved sync preview performance
  • Improved performance when fetching Redshift schema
  • Organizations could be created in an Affinity list during an update-only sync in some circumstances

rel2022.08.03

Fixes

  • Building models from Views could result in an error
  • Pasting a column from the Query Runner schema pasted fully qualified name into the query; only the local name is pasted now
  • Sync preview performance is improved
  • Server will fail to start if the Deployment and Key are not set

rel2022.08.02

Features

  • Added support for authorizing Postgresql connections using client certificates
  • Added support for connecting to Google CloudSQL Postgresql using an IAM service account
  • Added support for syncing Marketo Activities and Program Members
  • Added support for syncing Shipbob Product fulfillment line items

Fixes

  • ~Organizations in the Affinity global namespace were incorrectly identified as "updates"~
  • Organizations could be added to Affinity lists in an update-only sync
  • Affinity array values were not formatted using their element types before syncing
  • Organizations in the Affinity global namespace were included in Affinity model results
  • Basic auth header could be omitted in paginated requests in some circumstances

rel2022.07.25

Features

  • Added support for syncing Shopify data
  • Added history to Query Runner

Fixes

  • Text values were erroneously removed from sync configurations under some circumstances
  • Reduced Zendesk overhead when syncing users without organization membership
  • It was possible to have duplicate field mappings in a sync configuration, which led to ambiguous behavior
  • Single record syncs to Salesforce could fail under some circumstances
  • Hubspot association lookup errors could cause sync to fail, rather than reporting as an error
  • Significantly educed memory consumption for large syncs
  • Added support for a single operational bucket for all Polytomic data
  • Netsuite picklist fields were not mapped to their internal identifier during sync
  • Salesforce numeric fields did not have the full complement of filters exposed when used in a target filter

rel2022.07.19

Fixes

  • Improved performance of Query Runner SQL editor
  • Syncs interrupted by system shutdown will be restarted
  • Extraneous warnings were logged in some situations for syncs to Salesforce
  • Affinity test record links could result in a "not found" error
  • Enriching companies from Harmonic would result in long-running syncs in some situations
  • Alphabetize schema and field list in Query Runner
  • Test file created when create a sync to blob storage (S3, Azure Blob Store, Google Cloud Storage) was not cleaned up
  • When creating a model for a SQL connection with no tables available we now default to the View selector

rel2022.07.15.1

Features

  • Include environment variable to tune the garbage collector
  • Add ptctl command to modify sync feature flags

rel2022.07.15

Fixes

  • An extension is no longer required when creating a sync to a new file in S3
  • SSO configuration can be updated via the command line

rel2022.07.14

Features

  • Support for writing to Hubspot Line Items and Products

Fixes

  • BigQuery Projects and Datasets labeled correctly in Query Runner
  • Query runner will now insert at the cursor position instead of at the end of the query.
  • Temporary tables for BigQuery are now cleaned up properly if the job fails
  • Hubspot products and line-items (#2951)
  • A fix to the power-on test which could cause startup to fail

rel2022.07.13

Fixes

  • Athena connections could erroneously pass healthcheck
  • S3 connections now return the underlying errors encountered during connection
  • Salesforce warning counts could be incorrect
  • Added support for using an S3 bucket for job payloads

rel2022.07.12

Features

  • S3, Google Cloud Storage, and Azure can now be used to store job payloads.
  • Query Runner and Model Editor now support find and replace (use cmd/ctrl + F)

Fixes

  • Fixed a bug where the record log would silently coerce values.

Behavioral Changes

  • When multiple models are involved in a sync, the additional models are
    left-joined (to the identity model) instead of outer-joined.

rel2022.07.11

Features

  • Support for the Polytomic Terraform
    Provider
  • Improved performance and ease of use for Query Runner
  • Added support for globally disabling record logs
  • Users editing a sync are added to the error notification list

Fixes

  • Sync preview could fail when creating a sync to a new S3 file
  • Hubspot syncs could fail when models had many propreties
  • Running into our internal rate-limiter could cause Hubspot syncs to fail in
    some situations
  • Comparing fields of different but compatible types could cause SQL errors
  • Error retries may be disabled for individual syncs

rel2022.07.05

Features

  • Added support to command line interface for creating users from CSV files

Fixes

  • S3 bucket names are now handled consistently with the application; either
    s3://bucket, bucket, or bucket/prefix is now accepted everywhere
  • An inaccessible Athena catalog prevented any catalogs from being displayed
  • Retry execution errors button could return spurious error
  • A performance improvement when creating Marketo models

rel2022.06.30.01

Fixes

  • Only show authentication methods configured for a deployment
  • Do not retry errors from previous sync executions if the identity mapping has
    changed

rel2022.06.30

Features

  • Added support for using Stripe Payment Intents as a source
  • Added support for using Shipbob data as a source
  • When creating Airtable fields from Polytomic you can override the Airtable
    type (Airtable Enterprise only)
  • Added support for Airtable metadata as a source (Airtable Enterprise only)
  • Added support for running in ECS Fargate on premises

Fixes

  • Using an empty Harmonic watchlist as a model resulted in an error
  • Records with no non-empty fields could cause an error when synced to SFDC
  • Improved performance of introspecting Athena tables

rel2022.06.28

Features

  • Added support for syncing to customers in Kustomer
  • Added support for using customers and companies from Kustomer as models
  • Added support for using Stripe Accounts as a model
  • Added support for creating new Airtable tables & columns when creating a sync
  • Added support for OAuth client credentials flow with HTTP API connections

Fixes

  • The command line interface has been renamed to ptctl
  • Improved performance of enumerating Athena schemas and tables
  • Improved perforamnce of joining Companies & People from Harmonic Saved
    Searches and Watchlists
  • Improved handling of archived contacts when syncing to Intercom
  • Rate limit headers were not respected by HTTP API models
  • Match Intercome user emails case insensitively
  • Support using numeric custom properties in Hubspot for associations
  • Only one record was updated in Airtable when multiple matched the identity mapping
  • Table names displayed in Query Runner were uppercased
  • Null values were included with Salesforce payloads for relationship lookup by
    external ID fields, resulting in record errors

Behavioral Changes

  • Null value semantics were not correct for some sync destinations; by default
    Polytomic will not overwrite existing data with null values.
  • Polytomic will no longer attempt to traverse joins if the joined field value
    is empty.

rel2022.06.21.01

  • Retry button was not displayed in sync history table

rel2022.06.21

Features

  • Added support for setting a maximum offset when paginating through API results
  • Added record logging support for Zendesk syncs

Fixes

  • Malformed error CSV from Salesforce could cause the actual errors to be
    supressed
  • Reduced memory overhead for syncs to Snowflake
  • Test sync results had the identity field omitted in some situations
  • Number of warnings recorded for a sync is now capped

rel2022.06.13

Features

  • Add support for syncing to Harmonic watchlists
  • Add support for syncing to Google Cloud Storage
  • Add support for syncing to Azure Blob Storage

Fixes

  • Only changed records will be synced from models with tracking fields & a
    unique identifier set
  • API models could wait for responses indefinitely
  • Fixed compatibiltiy with MySQL 5.x
  • Relationships were unable to be set in some Salesforce syncs
  • Syncs from Cosmos could report credential error when reading from reading from
    regions

rel2022.06.07

Features

  • Add support for syncing from Survicate
  • Notifications are sent when a connection becomes healthy
  • Salesforce object models automatically set their unique identifier (Id)

Fixes

  • API limits could be set without specifying how to use them during requests
  • Deleted model fields are removed from syncs
  • Unsigned ints could fail to coerce types properly
  • Force resyncing an update only sync would truncate the destination table
  • Improved performance when creating BigQuery models
  • Hubspot validation errors could be erroneously reported as "invalid value"
  • Syncs using Harmonic company enrichment could miss changes that occured in
    Harmonic's data set
  • Users were unable to create models for some Opportunity-related Salesforce
    objects

rel2022.06.01

Features

  • Added support for syncing to S3
  • Added support for Marketo as a model
  • Added support for using all Harmonic Saved Searches as a model

Fixes

  • Null characters in model example values could produce an error when saving
  • Identity field value was omitted from Preview records in some situations
  • Syncs to Stripe could incorrectly double total record count

rel2022.05.24

Features

  • Added record log support to Airtable, Affinity,
  • Exposed Harmonic updated_at

Fixes

  • Default Salesforce request limit was not set when creating a new connection in
    some situations

rel2022.05.20

Fixes

  • Connections that were not used by a sync are now actively health-checked
  • OAuth tokens refreshed in the course of a health check are retained for future
    requests
  • In some situations Polytomic would attempt to refresh a previously expired
    token
  • Creating a new Snowflake schema could result in the display of spurious errors

rel2022.05.19

Features

  • Added support for custom rate limits on Zendesk connections
  • Added support for append-only syncs to Google Sheets

Fixes

  • OAuth tokens could expire before being refreshed in some situations
  • Updated record log was incomplete for Salesforce destinations in some
    situations
  • Syncs with model filters included the filter field value when de-duplicating
  • Some model filters failed when used with Postgres UUID fields
  • Pipedrive rate limit backoff was not respected in some situations
  • Errors from enumerating BigQuery schemas were not reported
  • Syncing numeric fields from BigQuery to Google Sheets resulted in values
    expressed as fractions

rel2022.05.01.01

Fixes

  • Fixed speed regression with condition evaluation during syncs
  • Salesforce syncs may not pull the latest token when a error is encountered

rel2022.05.11

Features

  • Added support for syncing to Dialpad Contacts
  • Added support for syncing to BigQuery Record and Repeated fields
  • API models support transforming the pagination token during requests
  • Model editor displays syncs which currently use the model

Fixes

  • Record logs did not include identity value for syncs to Redshift and BigQuery
  • Applying conditions Postgres UUID fields resulted in an error
  • Field mapping create-only mode was not respected for some sync configurations

rel2022.05.04

Features

  • Pipedrive Deals now supports fetching deleted records

Fixes

  • Improved error localization/messaging around the app
  • Google sheet models can now survive renaming
  • Create-only field in syncs now have create-only mode by default
  • Airtable models can be created even if the table is empty

rel2022.04.27

Features

  • API models can extract next-page URL from HTTP response headers

Fixes

  • Handle cases where Hubspot sends unexpected association data
  • SSH tunnels are now health-checked indepedently of the database connection
  • Reduced memory usage for most deployments

rel2022.04.21

Fixes

  • Fix a bug causing unnecessary connections to databases/caches

rel2022.04.19

Features

  • Added support for custom HTTP headers on Webhook connections

Fixes

  • Improvements to queueing when multiple syncs attempt to write to the same
    destination
  • Report more helpful errors from Snowflake syncs
  • Polytomic managed schemas will use uppercase field names for Snowflake
  • Non-numeric postal codes in Pardot could cause sync errors

rel2022.04.12

Features

  • SQL connections can now sync into any schema (as well as create new schemas)
  • Create/Update mode for Redshift
  • Support setting "Default Organization" on Zendesk Users.
  • Harmonic Watchlist support

Fixes

  • Various UI/accessibility improvements with focused fields
  • Intercom integration no longer double-counts records in sync summary
  • Various fixes and improvements for Netsuite

rel2022.03.29

Features

  • Added support for using Pipedrive as a source
  • User-defined fields may now use JSON path expressions

Fixes

  • Null values in numeric fields were previously expressed as 0 in some cases;
    they are now correctly expressed as null values.

rel2022.03.15

Features

  • Added Replace mode support to MySQL syncs
  • Added Chargebee Product Catalog 2.0 support
  • Added support for offset-based API pagination

Fixes

  • Tables were not truncated when performing a BigQuery full resync
  • Concurrent sync executions against a single destination could result in
    deadlock

rel2022.03.08

Features

  • Added support for Netsuite custom list fields
  • Added support for Affinity custom fields in models

Fixes

  • Handle additional error cases when writing to Affinity
  • Quotes in Intercom field values were incorrectly escaped
  • Improved error reporting for syncs to Redshift

rel2022.03.01

Features

  • Added support for Netsuite Employee models
  • Adjusted Snowflake & Redshift destinations for improved warehouse utilization

Fixes

  • MongoDB ObjectIDs are correctly emitted when used in a sync
  • Operations that return an error when writing to Snowflake will be retried

rel2022.02.22

Features

  • Improved performance when syncing to Snowflake

Fixes

  • Provide more informative error messages for query errors
  • Syncing from Cosmos could send unnecessary records in some cases
  • Sample values for Mongo array fields showed a single value, rather than the
    array
  • Harmonic enumerations are coerced to human readable values

rel2022.02.15

Features

  • Added support for connecting to Netsuite
  • Added support for duplicate partition key values in Cosmos collections
  • Exposed Harmonic emails & Highlights as formmatted strings
  • Added support for enriching company data from Harmonic based on website URL

Fixes

  • When creating a sync to a newly created table, Preview could fail
  • Records synced to Intercom could be counted twice in the total

rel2022.02.08

Features

  • Primary personal & work emails are now exposed for Harmonic persons
  • Syncs will create values in Airtable Select fields if needed

Fixes

  • Nullable UUIDs in Postgres databases could deserialize incorrectly in some
    situations
  • Salesforce connections handle expiration of tokens more gracefully
  • [Datetime expressions](https://docs.polytomic.com/docs/configuring-an-http-api-model#datetime-expressions-in-headers-and-query-parameters are now case insensitive
  • Retrying sync executions with a large number of errors could fail in some
    situations
  • Cosmos models will now retry requests when rate limited
  • The initial sync from Cosmos could be incomplete due to partitioning
  • SQL destinations will truncate the target table when forcing a full resync

rel2022.02.02

Features

  • Associations between Hubspot custom objects are now supported
  • Reduced requests needed when syncing Hubspot

Fixes

  • BigQuery sync failures could go unreported if failure occured after the "load"
    jobs
  • Numeric identity values could cause spurious duplicate warnings
  • Syncs to Snowflake could report inaccurate field counts
  • Empty strings could error when synced to an Airtable select field
  • "Does not contain", "does not end with" filters would evaluate to false when
    the value was empty
  • Numeric values could error when synced to Airtable under some circumstances
  • Google Sheets models could fail to detect column types

rel2022.01.26

Features

  • Added support for create+update syncs to Big Query
  • Airtable Record IDs may be used in sync identity mappings
  • Traction metrics for Harmonic companies may be added to models
  • Sync execution errors may be retried for Hubspot syncs

Fixes

  • Reading a blank padded character fields in Redshift could fail
  • Values are written to Google Sheets as user-entered values
  • Sync execution warnings were not exposed to the user in some situations

rel2022.01.18

Features

  • Add Hubspot Line Items as a model source
  • Add support for customizing API model fields

Fixes

  • MySQL JSON field values could be returned as strings rather than objects in
    some situations
  • Correctly sync Airtable duration fields
  • Hubspot syncs could trigger rate-limiting behavior
  • Do not attempt to enrich empty company value from Harmonic
  • Correctly trim Redshift padded character fields
  • Sync preview could fail if new fields were added to a Polytomic managed
    database target
  • Airtable errors were not linked to specific records in some cases

rel2022.01.11.01

Fixes

  • Only query Marketo for unique identity values

rel2022.01.11

Features

  • Add support for syncing to Redshift
  • Add support for enriching models with Harmonic company data

Fixes

  • Make Marketo search batch size configurable via MARKETO_BATCH_SIZE
    environment variable
  • An unhealthy connection could not be reconnected via the user interface in
    some situations
  • Retry syncs with non-string identity mappings could fail to complete
  • Is empty/Is not empty filters for numeric fields could construct invalid
    queries
  • Reduce sync concurrency to accommodate Zendesk job limits

rel2022.01.04.03

Fixes

  • Marketo auth token could expire while searching for matching Leads

rel2022.01.04.02

Fixes

  • Avoid "too few results" errors from Marketo

rel2022.01.04.01

(There is no rel2022.01.04 due to a build issue.)

Features

  • Add support for syncing to Pipedrive
  • Add support for using Dialpad aggregate reports and webhook call data as
    models

Fixes

  • Job locks could be held longer than necessary in some cases
  • Improve handling of syncs to Snowflake
  • Report ignored sync errors (ie, when ignoring invalid references) as rolled up
    warnings
  • Adjust request batching when Marketo reports too many possible results
  • Improve support for large Zendesk syncs
  • Improve support for associating Zendesk users & organizations

rel2021.12.23

New Features

  • Add support for replacing Postgresql tables during sync execution
  • Improve speed for syncs to Snowflake
  • A maximum sync duration may be set via the environment

Fixes

  • Display object values correctly in Query Runner results
  • "contains" filters for BigQuery models resulted in an error in some situations

rel2021.12.21

New Features

  • Add support for reading Dialpad call logs
  • Add support for syncing Airtable attachment fields
  • Add support for using Airtable formula fields in identity mappings

Fixes

  • Improved handling of Zendesk associations
  • Increased performance for syncs to Snowflake destinations
  • Invalid reference errors from Salesforce were not exposed sometimes
  • Sync failure notifications could be sent without an error shown
  • Some read-only fields were excluded from the sync record logs
  • Clustering keys are set when creating Snowflake tables
  • Newly created MySQL tables are prefixed with polytomic rather than being
    created in a schema

rel2021.12.03

Fixes

  • Don't fail partially successful BigQuery jobs

  • More efficient fetching of syncs list

  • Fix Snowflake batch inserter

  • Report an error if all records in SQL job fail

  • Support Snowflake VARIANT/ARRAY columns for create/update

  • Explicitly report timeouts to user rather than silently failing

  • Reduce over-fetching of SQL schemas

  • Correctly handle null arrays coming from Snowflake

rel2021.12.01

New features

  • Model filters for Cosmos DB

Fixes

  • More efficient Snowflake querying (async mode)

  • Fix Snowflake writes for arrays and JSON objects for replace syncs

  • Allow manual syncs to be selected for sync-after

  • More stability in the face of errors when writing to SQL databases

  • Fix 'Last modified' to only display on saved models

  • Fix memory leak on sync status page

  • Exclude read-only types from Airtable destinations

  • Performance improvements for sync history page

rel2021.11.24

New features

  • Show 'Last modified by' info on models and syncs

Fixes

  • Surface failures from Smartsheet jobs

  • Fix duplicate header in record logs

  • Correctly format dates for Google Sheet exports

  • Fix bug with connecting to Airtable bases

  • Improve performance of API model editor

  • Fix bug with per-field 'create-only' mode when applied to HubSpot destinations

  • Fix bug with some date formats when writing to Salesforce

rel2021.11.17

Fixes

  • More robust handling of Affinity source data

  • More robust writing to SQL databases when some records fail

  • Correctly handle embedded commas when syncing Google Sheets to Snowflake

  • Show 'Run after' syncs list on hover on syncs page

  • Remove 'Browse' button from saved Google Sheets connections

  • Enforce model refresh if query changes

  • Robust handling of large (300k+) simultaneous record errors when syncing

  • Decode Hubspot array fields correctly

  • More efficient browser memory usage when viewing sync history page

  • Faster load time in Query Runner for large schemas

  • Fix occassional 'unreachable fields' display bug in sync config

rel2021.11.10

New features

  • SQL Runner supports SOQL queries

  • Add 'Contains' model filter with arrays

  • Faster performance of SQL runner schema display with many tables

  • Add body parameter for API source models on POST

Fixes

  • Handle Hubspot array fields correctly

  • Improve memory pressure on sync history page

  • Correctly format dates for snowflake-destination replace syncs

  • Fix occasional 'unreachable fields' display bug

rel2021.11.04

New Features

  • Ability to define models from SOQL queries

  • Allow type overrides on API models

  • When writing to databases, automatically generate target column when new model field is added to sync

  • Add support for notifications on sync warnings

  • Record duplicate identity values as sync warnings

Fixes

  • More robust handling of nil values in Affinity

  • More robust handling of Airtable dates

  • Fix bug where per-field sync mode overrides weren't getting saved

  • Migrate to new HubSpot OAuth scopes

  • Remove limit parameter from API fieldsets

  • More robust handling of empty identity values

  • User setting toggle would sometimes be out of date

  • Do not fail sync on missing identity value

  • API sources now terminates pagination on duplicate results

  • Support datetime strings where offset is +/-hhmm (#1465)

rel2021.10.26

New Features

  • New source integration: Databricks

  • New destination integration: Smartsheet

  • Log Salesforce inserted and updated records

  • Log duplicate Salesforce records as warnings encountered during sync

  • Allow use of readonly Salesforce fields in identity mapping

  • Support for querying MongoDB

  • Support for ARRAY and BIGNUMERIC types in BigQuery

Fixes

  • Sync history record log now downloads as CSV rather then opening in browser as plaintext

  • Fix error when saving sync with null overrides

  • Fix connection error when connecting to Freshdesk

  • Faster querying of Google Sheets with many empty rows

rel2021.10.19

New Features

  • Enable string target filters for Salesforce

  • Enable logging to S3 of total records synced

  • Support for Microsoft 365 SSO

Fixes

  • Performance improvement for displaying large number of tables in SQL database model editor's table selector

  • On prem deploys can specify valid HTTP origins

  • Fail syncs with invalid configurations

  • Do not fail when a single record cannot be encoded

  • Performance improvement for large syncs to BigQuery

rel2021.10.13

New Features

  • CSV connections and models support headers and query params

  • Expose non-company entities for Harmonic

Fixes

  • Filter duplicate model mappings from override list

  • Better performance for dropdown menus

  • Fix rare case where a sync fails but still displays 'Running'

  • Better displaying of long strings in banner alerts

rel2021.10.06

New Features

  • Partial sync failure notifications

  • Syncing people objects from Harmonic

  • Support for generating models from SQL views

  • Expose Chargebee payment source fields in model editor

Fixes

  • More edge-case stability for Salesforce syncs

  • Fix parse error for SQL queries that contain URLs with a '?' character

  • Fix Chargebee edge case type errors

  • Smoother UI flow for Okta SAML

  • Fix occasional display of field ID rather than field name in sync preview

rel2021.09.28

New Features

  • Okta SAML login support

  • Much faster syncs from Stripe

  • Show errored record details for HubSpot sync errors

  • Expose Stripe Invoice and Invoice Line Item objects

Fixes

  • More thorough exposure of available fields for API models

  • More stability for HubSpot sync edge cases

rel2021.09.21.01

Fixes

  • Disable Salesforce composite API when a batch size is set

rel2021.09.21

New Features

  • New sources: AWS Athena, Azure Cosmos DB, Chargebee, Harmonic

  • New destination: Snowflake

  • Allow more than one sync override for the same model field

  • Faster interactions and saves for many-field models in model editor

  • When writing to SQL databases, Polytomic's auto-generated schema now includes created_at and updated_at columns

Fixes

  • Sort sync list by destination object name

  • Improve saving performance on model config

  • Optimisations for HubSpot syncs

  • Include Polytomic client ID when making Salesforce requests

rel2021.09.13

Fixes

  • Correct set HubSpot associations in more edge cases.

  • BigQuery schema generation now maps Arrays and Objects to String instead of Bytes.

rel2021.09.10

New Features

  • New sources: AWS Athena, Chargebee

  • Add ability to specify identity value in sync preview

  • Add ability to specify target filters (and filter logic) in addition to existing model filters

  • More efficient SFDC syncs

  • Show SQL database types in model editor

  • Include the execution type with webhook payloads

  • Add automatic verification for JSON path in API models

Fixes

  • Correct mapping of HubSpot associations in some cases

  • More robustness in the face of Google Sheet rate limits

  • Sort SQL table names in model editor dropdown

  • Enforce a platform limit for Salesforce calls

  • Fix mistaken identity values in some HubSpot cases

rel2021.09.01

New Features

  • New destination: Segment

  • Ability to add sync filters and overrides for API model fields

  • Ability to add filter logic for target fields, not just source ones

  • Ability to use datetime expressions in string constants

  • Ability to specify a minutes-threshold environment variable LONGSYNC_THRESHOLD for long-running-sync alerts

  • Add "Only sync if different" option for Salesforce syncs

  • Field-level create-only mode now will also write data if the record exists in the destination but the corresponding field is empty

  • Remove default last-24-hours filter on sync history page

  • Support for connecting to MySQL via SSH tunnel

Fixes

  • Fix URL to download error log for on-prem

  • Set a sync as failed if maximum number of retries is exceeded

  • Allow Marketo jobs to retry indefinitely

rel2021.08.18

New Features

  • Add ability to specify a constant string value in sync mappings

  • Add ability to override string values in sync mappings with user-entered value

  • Add ability to write to MySQL and PostgreSQL with auto-generated schemas

Fixes

  • History for continuous syncs now only lists syncs with non-zero updates

rel2021.08.11

New Features

  • New source: Azure Synapse

  • Sync filter predicates for strings, numbers, and dates

  • Sync filter logic

  • Filters for sync history page

  • Much faster loading of Snowflake tables in model editor

  • In-app documentation link

Fixes

  • Fix bug when syncing to Pardot checkboxes

  • More detailed error reporting for Salesforce edge cases

rel2021.07.27.03

Fixes

  • "Write null values" setting was not respected for Salesforce syncs in some
    situations

rel2021.07.27.01

Fixes

  • Changed records could be missed in some edge cases

rel2021.07.27

New Features

  • New destination: MySQL

  • Sync diffs now only consider the fields in a sync mapping rather than all fields in parent models

  • New sync schedule option: run after other syncs complete

  • Expose Hubspot enumerations as target values

  • Add support for Hubspot Tickets

  • Add additional types to Salesforce models (picklist, address, int)

Fixes

  • More generous Snowflake timeout when checking its connection

  • More generous SQL Runner timeouts for long-running queries

  • Fix SQL Runner scrollbar bug on Chrome for macOS

  • Fix SQL Runner empty-row results for BigQuery

  • Validate unreachable fields and invalid filters on sync save

  • More efficient connection health checks

  • Unadulterated display of Byte[] types from BigQuery

  • Expose Affinity ID for Person, Org identity

  • Affinity rate-limiting improvements

  • Fix weekly sync Sunday display bug

  • Move Google Sheet tab selection to model config

  • Extend Snowflake timeout when checking a connection

  • More efficient handling of simultaneous SQL database connections

  • More efficient handling of HubSpot rate limits

  • More efficient handling of Pardot rate limits

  • Normalize case of email addresses

  • Format empty arrays for Hubspot

  • Add additional types to salesforce mapping

  • Syncs page now orders them deterministically

  • Clearer error messages for API sources

rel2021.07.13

New Features

  • New-user onboarding flow

  • New source: CSV URL

  • New destinations: BigQuery, Klaviyo, and Google Ads

  • Add field-detail view to filters

  • Support non-equality filters on Stripe invoices

  • More efficient Salesforce syncing

  • More efficient syncing from Stripe

Fixes

  • More stable Salesforce token refreshes

  • More accurate detection of Salesforce external-ID errors

  • Fix some cases where filtering on Salesforce fields was not applied

  • Use correct encoding for Salesforce null values when mixing Bulk v2 and composite APIs

  • Correctly handle Salesforce fields without labels

  • Join target field in model query editor now also displays its model

  • Show connected user's email for Google Sheets connections

  • Show sync error count while hovering over partial-error icon

  • Pardot bugfixes

  • Sync history duration was inaccurate in some cases

  • Sanitize API source pagination

  • Support parameterizing arguments on API sources

  • Add showFieldDetails to sync filters

  • Only short-circuit processing w/no filters set

  • Set IsAssociation for all SFDC relationship ids

  • Sync summary mappings view now shows field details on hover

  • Show error message on unreachable sync filter fields

rel2021.06.29

New Features

  • New destination: Pardot

  • Display SFDC and Marketo API count even when user has not set a limit

  • Less SFDC API usage

Fixes

  • More efficient syncing from Stripe Customer record

  • Show response status code in webhook request log

  • Fix cases where error counts would be inaccurate

  • Fix cases where completed time would not update for SFDC Bulk v1

  • Store full request URL with webhook logs

  • Remove duplicate Lead errors if 'Skip converted' is set

rel2021.06.24.01

Fixes

  • Correctly enable Salesforce integration based on configuration

rel2021.06.24

New Features

  • Expose calculated fields on Salesforce destinations

  • Support boolean destination field filters for Salesforce

  • Hide unreachable fields in sync mappings

  • Use connection logos for sync summary field logos

  • Add field API name to sync 'Show details' view

  • Enable request logs for webhook syncs

  • Ability to add metadata to webhook syncs

  • Readable errors for failed HTTP API-source syncs

Fixes

  • More stability for Stripe joins

  • Nulls now copied correctly for SFDC Bulk V1 batches

  • Security improvements

  • Sync errors now consistently surface during retries

  • Correctly display nested BigQuery schemas in table picker

  • Decreased network load per connection

  • Show local field label in "Inbound joins" UI entry

  • Hide sync filters if mappings are empty

rel2021.06.08.02

  • Uploads to Marketo were erroneously marked as failed, regardless of final
    status.

rel2021.06.08.01

  • Log job & failure information from Marketo

rel2021.06.08

New Features

  • New destination: Freshdesk

  • Ability to clone syncs

  • Show timestamp of when a connections objects were last retrieved

  • Token pagination for REST API sources

Fixes

  • Correctly infer array and object model fields

  • Fix instability when unable to retrieve Zendesk custom fields

  • Platform support for denormalized records (#745)

  • Add support for Token pagination for APIs

rel2021.06.01

New Features

  • Ability to sync from arbitrary APIs

  • Ability to sync to arbitrary webhooks

  • Ability to trigger syncs through API

  • Ability to add all fields from a model in sync mapping in one click

  • Show field types in model query editor

  • Support for syncing multi-select strings to HubSpot

  • New 'completed with errors' status in sync history page

Fixes

  • Increased efficiency of cross-model joins

  • Increased stability of MongoDB connections

  • Order models by name in sync mapping dropdown

  • Better support for Stripe sub-fields in model builder

rel2021.05.25

New Features

  • Add date filter to sync filters

  • New destination: Webhooks

  • New source: HTTP APIs

rel2021.05.18.05

Fixes

  • Upgrade Snowflake client library
  • Add support for Big Query NUMERIC fields

rel2021.05.18.04

Fixes

  • Expose SQL errors which could lead to premature query termination
  • Add support for setting region on bigquery clients
  • Reduce memory consumption used for duplication record detection

rel2021.05.18.03

Fixes

  • Add diagnostic logging for records excluded from syncs based on filters,
    change detected, and de-duplication

rel2021.05.18.01

Fixes

  • Fix situation where a Salesforce error could result in the sync halting

rel2021.05.18

New Features

  • New destination: Facebook Ad Audiences

Fixes

  • Fix sporadic instability with Salesforce syncs

  • Better type handling for Iterable syncs

  • Improved logging of Salesforce integration

  • Fix occasional error with HubSpot associations

rel2021.05.12

New Features

  • New destinations: Iterable and Amplitude

  • Weekly syncs summary email

  • Add ability to associate records in HubSpot syncs

  • Add support for 'does not equal' sync filtera

Fixes

  • 8x speedup for Snowflake queries using multiple tracking fields

  • Fix issue where scheduled syncs sometimes don't run

  • More stability for HubSpot syncs

rel2021.05.04.01

Fixes

  • Fix daily-schedule bug that recorded wrong minute value

rel2021.05.04

New Features

  • New sources: HubSpot and Stripe

  • Add null/not null sync filters for Salesforce target fields

  • Add sync filters for Google Sheets model fields

  • Show full database schema in SQL Runner

  • Add create and update-only sync modes for HubSpot

  • Expose equality sync filters for Hubspot model fields

Fixes

  • Faster loading of model and sync pages

  • Format dates properly when syncing to HubSpot

  • HubSpot test syncs sometimes did not show the correct records

  • Miscellaneous UI improvements

rel2021.04.27.01

New Features

  • Tracking fields are now optional

  • New UI design

  • SQL runner: run ad-hoc SQL queries on your databases

  • Ability to set create-only mode per-field

  • Show/hide field details in sync mapping config

Fixes

  • Hard timeout for sync preview operations that take too long

  • Show on-premises execution log URL for any started execution

rel2021.04.21

Fixes

  • Show on premises log link for any started execution

  • Fix occassional stability issue with Postgres connections

  • Better performance for large Airtable tables

  • Fix bug where a connection would mistakenly save on successfull OAuth

  • Expose Affinity CRM relationship fields for sync

  • More stable recovery in the face of errors from Google Sheets's API

  • Prevent occasional record duplicates in HubSpot syncs

  • Better support for Airtable formula fields

rel2021.04.13

New Features

  • Add support for syncing to Intercom companies

  • Add support for syncing from Airtable Views

Fixes

  • Sync field mappings are now listed in model-field alphabetical order

  • Faster fetching of Airtable bases and tables

  • Improvements for syncing to Affinity CRM

  • Redirect user to login on session expiration

  • Improved display of BigQuery table names

  • More stability when refreshing Google Sheets connections

rel2021.03.30

New Features

  • Cron sync scheduling via 'Custom' schedule option

  • Ability to sync to Stripe Customers and Subscriptions

  • Unhealthy-connection destinations are now marked as such on the syncs list page

Fixes

  • Security upgrades

  • Query speed improvements

  • (on-prem) More efficient database space utilisation

rel2021.03.17.02

  • Handle additional errors during Salesforce bulk job creation

  • Reduce the number of SQL queries needed when joining fieldsets

rel2021.03.17.01

Fixes

  • Optimize SQL queries when multiple tracking fields are used

  • Improve caching behavior for queries executed during a sync execution

rel2021.03.17

New Features

  • New destinations: Intercom, Affinity CRM

  • Syncs page is now grouped by destination

  • Model page query fieldsets are minimized by default

  • Expose support for sync field filters on Salesforce fields

Fixes

  • Improve load speed for syncs and model pages

  • Lower memory usage during syncs, especially for large ones

  • More efficient related-object syncing in Salesforce

  • Do not return model page error if we cannot enumerate SFDC schema

  • Google Sheets syncs would fail in rare conditions

  • Add ability to ignore invalid references

  • Better automatic model validation when executing a sync

  • Clean-up relationships from removed fields

  • Do not attempt to sync records we were unable to map to destination

  • Gracefully handle search errors from Hubspot

rel2021.03.02

New Features

  • New sync destination: Airtable

  • Syncs page now groups syncs by destination

  • Make query name sticky when scrolling through model fields selector

  • Add join indicators on model page

Fixes

  • Syncing MongoDB to Google Sheets would sometime fail

  • Cancelling a Salesforce sync from Polytomic after Salesforce has already written all the data should not count as a complete job

  • Show error message for the specific case of failing to update a Google Sheet

  • Expose Salesforce IDs as identity mapping targets

  • Security improvements

rel2021.02.23

New Features

  • Ability to set filters on model fields in a sync configuration: is null/not-null and true/false for boolean fields.

  • Banner confirmation of invitation email when a new user is added

rel2021.02.16.04

  • Retry network requests in the event of unexpected hang up

rel2021.02.16.03

  • Username and password were not always passed during the Redis client
    construction

rel2021.02.16.02

  • Errors when searching Marketo could result in a stopped sync

rel2021.02.16.01

  • Connections to SQL databases were prematurely removed from the connection
    pool.

rel2021.02.16

New Features

  • Expose 'sync all records' setting for all targets, not just Salesforce

  • New sync source: Airtable

  • Support for syncing to and from very large Google Sheets

  • Allow and respect update-only and create-only fields in a create-update sync

  • Indicate a Salesforce connection as unhealthy if Salesforce has run out of API calls for the day, even if that has nothing to do with Polytomic

  • Update weekly sync summary email to summarize the whole week

Fixes

  • Occasional inability to change a connection name

  • Edge cases that would result in inaccurate updated-record counts in the sync history

  • Introduce restriction of query relationship targets to existing model fields only

  • Speed optimisations for large-data syncs

  • Occasional errors for calculated columns in Google Sheets

rel2021.02.03.02

  • Update upstream Redis library to incorporate Redis Cluster fixes

rel2021.02.03.01

Fixes

  • Tracking field names were not always quoted correctly

rel2021.02.03

New Features

  • Implemented a weekly sync report

Fixes

  • Snowflake field labels are now title-cased rather than uppercase

  • Accurate indication of whether Salesforce relationship target fields are creatable or updateable

  • Fix some cases where cancelling a running sync would result in nothing

  • Better handling of default Salesforce batch sizes

  • More efficient Salesforce upserts in bulk mode

rel2021.01.26.02

  • More aggressively cache Marketo target definitions

rel2021.01.26.01

Fixes

  • Marketo syncs were not using the maximum available batch size

rel2021.01.26

New Features

  • Clickable download links for sync logs in on-premise deployments

  • Allow setting syncs to always be full syncs

  • Show Salesforce account user in its connection configuration

Fixes

  • Speed optimisations when gathering data for a sync

  • More efficient Salesforce API usage for some sync types

  • Further reduction in logging volume

  • Treat automatically-skipped syncs as failures

  • Treat Salesforce identity keys as case insensitive

rel2021.01.19.02

  • Remove logging statements without level filter

rel2021.01.19.01

Fixes

  • Updating Salesforce related objects using an external ID would sometimes fail
    when serial concurrency or fixed batch size was set.

rel2021.01.19

New Features

  • Add support for update-only or create-only Marketo syncs

Fixes

  • Creating a new Google Sheets connection would sometimes fail

  • Security upgrades

rel2021.01.17

Fixes

  • Marketo syncs could fail to start due to overly aggressive locking.

  • A sync could fail the first time it was run after changing referenced tracking
    fields in the model.

rel2021.01.12.3

Fixes

  • Fixed regression in model query editing

rel2021.01.12.2

Fixes

  • Relationships to related Salesforce objects were not set in some
    circumstances.

  • Preview was unavailable when a sync included related records.

  • Lookup fields on related records were erroneously flagged as required.

rel2021.01.12.1

Fixes

  • Corrects a bug which could cause duplicate records to be created in
    Marketo.

rel2021.01.12

New Features

  • Add ability to sync to any SFDC object's related-object's ID field

  • Add AzureSQL support

Fixes

  • Don't disappear 'full resync' button during sync

  • Allow the same model field to sync to two different target fields

  • Remove character limit on query titles

  • Stability improvements for particular Redis environments

  • Skip data fields for a particular sync if source system is unavailable

  • Editing query fields propagates those changes to
    relationships using those fields

  • Fix error preventing logging in to Polytomic Front widget in Front's desktop app

  • Security updates

  • Logging improvements

rel2021.01.05

New Features

  • Add support for Azure SQL connections

Fixes

  • Namespace all Redis keys for cluster mode compatibility; this includes the
    contents of all rel2020.12.23 hot fixes.

  • Update Docker base image to use Ubuntu 20.04.

  • Removing a fieldset involved in a relationship could result in the Model
    refusing to load.

  • An error during OAuth connection could result in a half-configured connection.

  • Hubspot syncs could drop the final record from a batch.

rel2020.12.23.4

Fixes:

  • Route redis scripts to the correct cluster node.

rel2020.12.23.3

Fixes:

  • Reference frontend scripts using a domain relative URL

rel2020.12.23.2

Fixes:

  • Set the Redis namespace for registered task handlers for Redis Cluster
    compatibility

rel2020.12.23.1

Fixes

  • Scope event stream keys correctly for Redis Cluster compatibility

rel2020.12.23

New Features

  • Add support for Amplitude model

  • Ability to scroll left and right sides of model edit view independently

Fixes

  • Display logout button in application sidebar

  • Edge-case stability for Zendesk syncs

  • More generous timeout for testing new DB connections

  • Empty Google Sheet results could fail the sync

  • A deleted identity field could cause the sync list fetch to silently fail

  • Allow adding a previously-removed user to an organization

  • Remove extraneous "Google Sheets" default name prefix from Google Sheet connections

  • Accept full domains or URLs for Zendesk connection domain

  • More stable handling for query tests that return an error

rel2020.12.15

New Features

  • Add support for syncing to Google Sheets

  • Add support for syncing to Zendesk

  • Add option to force a fresh authentication for existing OAuth connections

  • Expose Salesforce external ID fields for sync identity mapping

Fixes

  • Only display writeable target fields in mapping drop-down

  • Handle widget view empty state

  • Refreshing a database table schema in a query after editing field properties would revert said properties

  • Deleting fields used in an existing sync could prevent new syncs from being created

  • Deleting errored SSH-tunnel database connections would fail

rel2020.12.09

New Features

  • Added support for 'views': the ability to display a live subset of the Universal Model based on context (support ticket or customer email) in widgets that can be embedded in third-party customer systems like Front, Salesforce, and Zendesk. Front support only for now.

  • Added support for a substring-of identity mapping function in addition to the default equality one.

Fixes

  • Salesforce syncing would fail for certain field mapping type combinations

  • Under certain conditions, test syncs resulted in an inaccurate record log.

  • Editing a connection name sometimes resulted in the old name persisting.

rel2020.11.25

New Features

  • Syncing an array of values or delimited string to a multi-picklist
    field in Salesforce will set the picklist values.

    When syncing a multi-picklist field in Salesforce, comma,
    semi-colon, and null delimited strings will select picklist values,
    if a match exists. Syncing arrays to a multi-picklist field will
    alow select the picklist values.

  • Sync filters may be applied to Update syncs.

  • Updated BigQuery integration to support schema browser.

    Previously only SQL queries were supported for BigQuery fieldsets.

Fixes

  • Deleting a fieldset that is referenced by a sync caused the sync to
    disappear from the list.

  • When running on premises, duplicate root users could be created at
    startup.