We are running a nestjs-based monorepo containing a couple of services each having at least two instances in the google cloud k8s. Recently we implemented a couple of scheduled jobs to munch some data stored in a mongo replica set and spit out the results every 10 minutes. What we did not think of is that having multiple instances of the same service will result in running the same job as many times as we have instances at the same exact point in time, against the same exact data set. What is the best practice to handle that and have the scheduled job run in only one instance, only once at the predefined time?
NestJS - Task Scheduling - Prevent running the same Job in parallel in identical service instances in K8s
Viewed 2211
1 Answers
You need to use the .spec.concurrencyPolicy field in your Cron Job spec and set it to Forbid, to prevent concurrent runs of the same job.
See the Concurrency policy chapter in this k8s doc: https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/
Related