How to move file to a sublevel in s3 without triggering lambda?

userB4J71 0
Viewed 9

An excel file (say my_excel_file.xlsx) will be uploaded to s3://my-bucket/a/b/

A trigger is set in lambda with following properties: bucket-name: my-bucket, prefix: a/b/

I want my lambda to:

  1. Read the excel file uploaded to s3://my-bucket/a/b/ into a pandas dataframe
  2. After processing it, move the excel file to s3://my-bucket/a/b/archive/ with the name: my_excel_file_timestamp.xlsx

In case I am able to achieve the above step, will the lambda get invoked recursively? If yes, is there a workaround?

amazon-s3 aws-lambda amazon-web-services
1 Answers

Since Amazon S3 event is configured to trigger on prefix a/b/, then it will trigger the AWS Lambda function when an object is placed into a/b/archive/.

I recommend adding a line of code at the top of the Lambda function that checks the Key, which is passed to the function via the event parameter. It should check if the Key starts with a/b/archive/ (or similar rule) -- if so, it should exit the function immediately. This will not incur a significant cost because it will exit quickly and Lambda is only charged per millisecond.

The alternative is to put your archive folder in a different location.

John Rotenstein0
Related