AWS DynamoDB

Source

  1. In Polytomic, go to ConnectionsAdd ConnectionDynamoDB.
  1. Enter the following information:
  • AWS Access ID
  • AWS Secret Access Key
  • AWS region (e.g. us-west-1)
  1. Click Save.

IAM Permissions

Polytomic uses the following IAM Actions when reading from DynamoDB:

  • dynamodb:BatchGetItem
  • dynamodb:Describe*
  • dynamodb:List*
  • dynamodb:GetItem
  • dynamodb:Query
  • dynamodb:Scan
  • dynamodb:PartiQLSelect

AWS's documentation lists some example policies here.

Additional permissions when using DynamoDB Streams

When the "Use DynamoDB Streams for bulk syncs" checkbox is set, a bulk sync will enable a DynamoDB Stream for each table being synced and use that to read changes.

The following IAM action must be allowed for tables; this allows Polytomic to manage the DynamoDB stream:

  • dynamodb:UpdateTable

The following IAM actions must be allowed for on streams, in order to read the changes:

  • dynamodb:GetRecords
  • dynamodb:GetShardIterator
  • dynamodb:DescribeStream
  • dynamodb:ListStreams

These need to be allowed for the stream resource, not the table. If you're specifying resources the identifiers follow the pattern "arn:aws:dynamodb:us-west-2:123456789012:table/tableName/stream/*", where tableName is the name of individual DynamoDB tables to allow. See the AWS documentation for more information.

Syncing from DynamoDB

Warehouse and cloud storage destinations

Use Bulk Syncs when syncing from DynamoDB to your data warehouses, databases, and cloud storage buckets.

We recommend enabling CDC (change data capture) streaming from DynamoDB's Streams, as this avoids full table scans and instead reads updates from DynamoDB's changestream. You can enable this by ticking the Use DynamoDB Streams for bulk syncs option on your DynamoDB connection:

SaaS application destinations

Use Model Syncs when syncing DynamoDB to SaaS applications like Salesforce, HubSpot, Zendesk, and so on.

Considerations when using Model Syncs from DynamoDB

Secondary index

When building a Polytomic model on a DynamoDB table with a secondary index, you have the option of pointing Polytomic to your secondary index using the Index dropdown. Doing so will avoid full table scans:

Strongly consistent reads

By default, DynamoDB's read model is an eventually consistent one. That is, the response for a read operation may not include the result of a recently-completed write operation. This keep read throughput high.

You can override this within Polytomic. To have Polytomic carry out strongly consistent reads, tick the Strongly consistent reads box in your Polytomic model editor. The tradeoff is that your reads will take a bit longer: