AWS - Archive TTL deleted data from DynamoDB to S3

Viewed 398

I want to archive data from DynamoDB to S3. I came across two solutions:

  1. DynamoDB -> TTL DynamoDB + DynamoDB Stream -> Lambda -> Kinesis Firehose -> S3
  2. DynamoDB -> TTL DynamoDB + DynamoDB Stream -> Lambda -> S3

Which option is better and why? What are the advantages of using Kinesis Firehose and the disadvantages of not using Kinesis Firehose?

1 Answers

Using Firehose gives you some more options to configure how the data gets into S3. For example, since S3 bills on a per API call, you could consolidate 10 item deletes from the DynamoDB Stream into 1 write to S3 thus saving money. You can also do minor transformation with Firehose, if that is something you need.

Also, when writing your lambda function, make sure you grab just the TTL deletes filter on the value of type being Service. If you do this, you'll only get the TTL deletes, not normal deletes as normal deletes have a user account attached to them, TTL do not, thus the Service value.

Related