I'm using Celery to distribute multiple long running tasks. The celery worker is launched inside a container and scaled using docker-compose.
My goal is to be able to handle a clean shutdown on the reception of a signal like SIGINT, where a running task would be able to execute some code and quit, without waiting the end of it's execution.
An example would be the following :
@app.task(ignore_result=True)
def test():
try:
# Long running code
import time
time.sleep(1000)
except SystemExit:
# Store Progression
print('Exit !!!!!!!!!')
When I'm not using Celery, I'm able to catch the signal SIGINT and raise a SystemExit with the following code :
import signal
import sys
def sigterm_handler(_signo, _stack_frame):
# Raises SystemExit(0):
sys.exit(0)
signal.signal(signal.SIGTERM, sigterm_handler)
With Celery I'm only able to catch the signal at the worker level, but I'm left with no way to share this signal with a running task.
Other answers indicate the need to use worker_shutting_down, but I've still the same issue to comminicate with the running tasks.