Send an exception to a running long task to cleanly stop with celery

Viewed 68

I'm using Celery to distribute multiple long running tasks. The celery worker is launched inside a container and scaled using docker-compose.

My goal is to be able to handle a clean shutdown on the reception of a signal like SIGINT, where a running task would be able to execute some code and quit, without waiting the end of it's execution.

An example would be the following :

@app.task(ignore_result=True)
def test():
    try:
        # Long running code
        import time
        time.sleep(1000)
    except SystemExit:
        # Store Progression
        print('Exit !!!!!!!!!')

When I'm not using Celery, I'm able to catch the signal SIGINT and raise a SystemExit with the following code :

import signal
import sys

def sigterm_handler(_signo, _stack_frame):
    # Raises SystemExit(0):
    sys.exit(0)

signal.signal(signal.SIGTERM, sigterm_handler)

With Celery I'm only able to catch the signal at the worker level, but I'm left with no way to share this signal with a running task.

Other answers indicate the need to use worker_shutting_down, but I've still the same issue to comminicate with the running tasks.

0 Answers
Related