DynamoDB database model for storing different objects

Viewed 644

I'm trying to learn DynamoDB just for didactic purposes, for that reason I propose myself to create a small project to sell vehicles (cars, bikes, quad bikes, etc) in order to learn and get some experience with NoSQL databases. I read a lot of documentation about creating the right models but I still cannot figure out the best way to store my data.

I want to get all the vehicles by filters like:

  • get all the cars not older than 3 months.
  • get all the cars not older than 3 months by brand, year and model.
  • And so on the same previous queries for bikes, quad bikes, etc.

After reading the official documentation and other pages with examples (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-general-nosql-design.html#bp-general-nosql-design-approach , https://medium.com/swlh/data-modeling-in-aws-dynamodb-dcec6798e955 , Separate tables vs map lists - DynamoDB), they said that the best designs use only one table for storing everything, so I end up with a model like the next below:

-------------------------------------------------------------------------------------
Partition key |          Sort key     | Specific attributes for each type of vehicle
-------------------------------------------------------------------------------------
    cars      | date#brand#year#model | {main attributes for the car}
    bikes     | date#brand#year#model | {main attributes for the bike}
-------------------------------------------------------------------------------------

I've used a composite sort key because they specify that is a good practice for searching data (https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-sort-keys.html).

But after defining my model I end up that the previous model will have a problem called "Hotspotting" or "Hoy key". (https://medium.com/expedia-group-tech/dynamodb-data-modeling-c4b02729ac08, https://dzone.com/articles/partitioning-behavior-of-dynamodb) because in the official documentation they recommend having partitions keys with high cardinality to avoid the problem.

So at this point, I'm a little stuck about how to define a good and scalable model. Could you provide me some help or examples about how to achieve a model to get the queries above mentioned?

Note: I also considered creating a specific table for each vehicle but that would create more problems because to find the information I would need to perform a full table scan.

1 Answers

A few things...

hot partitions, only come into play if you have multiple partitions...

Just because you've got multiple partition (hash) keys, doesn't automatically mean DDB will need multiple partitions. You'll also need more than 10GB of data and/or more than 3000 RCU or 1000 WCU being used.

Next, DDB now supports "Adaptive Capacity", so hot partitions aren't as big a deal as they used to be. why what you know about DynamoDB might be outdated

In connection with the even newer "Instantaneous Adaptive Capacity", you've got DDB on demand.

One final note, you may be under the impression that a given partition (hash) key can only have a maximum of 10GB of data under it. This is true if your table utilizes Local Secondary Indexes (LSI) but is not true otherwise. Thus, consider using global secondary indexes (GSI). There's extra cost associated with GSIs, so it's a trade off to consider.

Related