When cronjobs are set to replace, does Kubernetes wait for the previous job to finish shutting down before starting the new one?

user_12460181 0
Viewed 1205

Just as the question says, I'm just looking to know if Kubernetes waits for some kind of confirmation from the previous cronjob that it has completely stopped before starting the new one, or if the kill signal is sent and the new job is started at the same time.

For reference, here is all the docs say on the replace policy:

https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#concurrency-policy

Concurrency Policy

  • Replace: If it is time for a new job run and the previous job run hasn’t finished yet, the cron job replaces the currently running job run with a new job run
kubernetes-cronjob kubernetes
1 Answers

Short intro/definition

  • Cron jobs go back long in the history of UNIX and Linux. Combined with other Kubernetes technologies like Pods, containers, the scheduler, and the intelligent algorithms for Pod placement and health probes, CronJobs prove to be way more powerful than their traditional OS-level counterparts.
  • Since they run on containers, CronJobs provide a lot of flexibility for the developers. They need not worry about which platform the cron job runs on and the presence of the required dependencies as everything runs on the container.
  • Kubernetes handles CronJob execution, what happens when it misses an execution time, and how many times the job should run. This allows the developers to focus more on writing code and addressing business issues rather than worrying about the internals of code execution.
  • The business application is still responsible for handling what happens when the cron job runs, does not run, gets canceled, or runs concurrently.

Read more here: cronjob.

Referring to your question when a CronJob has a concurrency policy set to Replace and it is still working, the Job will be deleted and it also deletes the Pod - look at the code.

While a Pod is deleting the Linux container/s will be sent a SIGTERM and then a SIGKILL after a grace period, default set to 30 seconds. The terminationGracePeriodSeconds property in a PodSpec can be set to override that default.

Look at the flag in code which is added to the DeleteJob function. This seems that this delete is only deleting values from the kube key/value store. This could mean the new Pod or Job could be created while the current Pod or Job is still terminating. You could confirm with a Job that doesn't respect a SIGTERM and has a terminationGracePeriodSeconds set to a few times your clusters scheduling speed.

Take a look on @Matt's answer: cronjob-shuttingdown.

user_113003820
Related