When Celery is staring up, it states that it has a backend like this:
- ** ---------- .> app: app.celery:0x7f5fe5f80b80
- ** ---------- .> transport: redis://redis:6379/10
- ** ---------- .> results: redis://redis:6379/11
- *** --- * --- .> concurrency: 4 (prefork)
But then when I try to run a bit of a complex task, I get this error:
NotImplementedError: Starting chords requires a result backend to be configured
Celery is configured like this:
Celery(__name__, broker="redis://redis:6379/10", backend="redis://redis:6379/11", tasks_acks_late=True)
Tasks are functions decorated with @app.task(base=MyBaseTask, bind=True), and the top level task is constructed in a way that there will be chords in there. The top level task is executed as my_task.delay(), and this is the line that throws the exception.
I understand why I need result backed to run my task, but I don't understand why it thinks that one isn't configured.
Any idea what I could be missing here?
EDIT: It might be important to note that I have the same codebase deployed on two k8s pods, one as an API server and one as a Celery worker. I'm calling .delay() on the API server and expecting the worker to pick up the task and run it. It is call to .delay() on the API pod that is throwing the exception. Each of the two pods instantiates its own Celery app, so I'm wondering if that is causing my problem?