Choosing Primary key for DynamoDB

user_11285570 0
Viewed 68

A bit of context: I am trying to build an inventory to list my AWS resources in various accounts and I am planning to use DynamoDB to store the data. These will be the columns for my table: ResourceARN, ResourceName, ResourceType, StandardTag, IsDeleted, LastUpdateTime and ResourceCreationDate ( this field is available only for a few resource types like Ec2).

Question: I want to query my DDB table using account ID, resource type and tag name. I am stumped on choosing the primary key for the table. Since primary key should be unique and has to have 1:many relationship. Hence, I cannot use a combination of resourceType and account Id. Nor can I use resourceArn as my primary key since it is 1:1 relationship. Also, using the resourceARN as the sort key does not make sense to me. I understand that I can use a simple scan operation, but that is very costly and will take time if I add more data in my DDB.

I would appreciate any suggestions or guidance over the same.

amazon-dynamodb primary-key data-modeling
1 Answers

Short answer

  • Partition key: Account ID
  • Sort key: <resource type>/<resource ID>

Rationale

It's a common pattern for a sort key to be a string concatenating multiple attributes. Since sort keys can be queried by prefix, you can leverage this in your queries:

  • Get all account resources: query all sort keys on the Account ID partition key
  • Get all EC2 instances of an account: query with partition key = <your account ID> and sort key begins_with('ec2-instance').

You may notice that ARNs follow such a hierarchy as well (what's probably not a coincidence). This would be effectively using a subset of the ARN as the sort key.

Some notes:

  • DynamoDB is about attributes as much as about columns. You don't need to include ResourceCreationDate in the records which don't have it, and doing so will save you space (see next point).
  • Attribute names count as storage for every record, which impacts cost and also throughput. It's common to use shorthand for names for this reason (rct instead of ResourceCreationTime for example).
  • You can use LSIs (Local Secondary Indexes) to order by creation and update times if you need this.
user_893030
Related