Prevent lambda from processing dynamodb stream events during table recovery

cloudy-cloud 0
Viewed 19

I'm working on preparing a disaster recovery plan for DynamoDB. In a DR situation we would create a temporary table to restore a snapshot to. From the Temp table we would copy data to a table that has been provisioned with IAC. Our DynamoDB tables have a stream and associated lambda trigger which would process all events that were copied in from the data in the temp table, this is unwanted and would cause a bunch of downstream issues.

Ideally I would like to disable the stream/ lambda trigger until the restore is complete and then enable and not process/ignore any of the changes from the copy/restore process.

I've read through DynamoDB stream documentation and it isn't clear to me if disabling the stream will clear events. It is my understanding that disabling/ enabling the DynamoDB stream although will provide you with a new arn is still the same stream behind the scenes and that log lives for 24 hours and once enable events would be sent to the lambda trigger.

Seems I might be able to configure the trigger on the lambda side, disable it, and then set the ShardIteratorType to 'LATEST' in order to prevent reading events from copied data.

Thanks in advance for any advice.

amazon-dynamodb-streams aws-lambda amazon-web-services
1 Answers

For anyone traveling down this path in the future I received some good answers on AWS Re:post here.

Some take aways:

  • When you disable/enable a stream it changes ARN and is completely separate to the old stream.
  • If the stream is disabled and you copy data to the table the stream log will remain empty once you reenable.
  • lambda event source mapping solution: LATEST will start reading from the the point of when you enable the trigger on the ESM. So if your copy puts events on the stream and you later create an ESM with iterator position as LATEST your Lambda will disregard all the data from the copy.
cloudy-cloud0
Related