AWS Lambda vs AWS step function

user_3221848 0
Viewed 22040

I am designing an application for which input is a large text file (size ranges from 1-30 GB) uploaded to S3 bucket every 15 min. It splits the file into n no of small ones and copy these files to 3 different S3 buckets in 3 different aws regions. Then 3 loader applications read these n files from respective s3 buckets and load the data into respective aerospike cluster.

I am thinking to use AWS lambda function to split the file as well as to load the data. I recently came across AWS step function which can also serve the purpose based on what I read. I am not sure which one to go with and which will be cheaper in terms of pricing. Any help is appreciated.

Thanks in advance!

aws-step-functions aws-lambda amazon-web-services amazon-s3
2 Answers

Lambda and Step functions are like floors and steps to each floor. You cannot replace one with another.

Lambda is computing, steps functions take them to the desired step.

Youtube video explains very well: https://www.youtube.com/watch?v=Dh7h3lkpeP4

To the analogy again, you can have multiple computes (lambda) in a single floor before you pass it on the next floor.

One of the example is as shown below.

Usecase: https://john.soban.ski/transcribe-customer-service-voicemails-and-alert-on-keywords.html

enter image description here

Hope it helps.

user_2994620

Step functions are excellent at coordinating workflows that involve multiple predefined steps. It can do parallel tasks and error handling well. It mainly uses Lambda functions to perform each task.

Based on your use-case, step functions sound like a good fit. As far as pricing, it adds a very small additional charge on top of Lambdas. Based on your description, I doubt you'd even notice the additional cost. You'd need to evaluate that based on the number of "state transitions" you would be using. Of course, you'll also have to pay for your Lambda invocations.

user_42311100
Related