What is the best way to schedule a python script with AWS?

Juan Almada 0
Viewed 40

I have a question that I would like to solve:

I have 4 python scripts as follows:

  • main.py
    • script1.py
    • script2.py
    • script3.py

Scripts 1, 2 and 3 are run invoked in the main.py.

What I need is to be able to easily schedule this main.py to run once a week.

What AWS services would be best for this? From the architecture side I don't know much.

Thank you!

schedule aws-event-bridge aws-lambda amazon-web-services python
2 Answers

You can deploy the script in lambda if your execution time is under 15 minutes plus cloudwatch events for scheduling

For execution scripts > 15 minutes, I would suggest using AWS Batch to have the script on schedule on any of the supported compute environment like ECS or EC2

omuthu0

For something running once per week, you clearly do not want to have any infrastructure running continuously. This leaves AWS Fargate (good for containerization) or AWS Lambda (good for scripts that run in less than 15 minutes).

Based on the limited information you have provided, AWS Lambda would be a good option since it is simple to schedule.

You would need to change the scripts slightly to fit in the AWS Lambda environment. The Lambda 'function' is triggered via a call to a particular function you nominate. Since you have multiple scripts, you would need to determine whether to combine them all into the one Lambda function, or whether to have them each as a separate function and have the 'main' Lambda function call the other Lambda functions.

AWS Lambda also supports parallel runs by deploying multiple Lambda functions. This could be useful if you have a lot of work to perform, or you can ignore it and just run your code as a 'single thread'.

John Rotenstein0
Related