celery result_backend with postgresql - null for some fields in the task table

user_11612477 0
Viewed 526

I am using Celery for a POC. My objective is to create a workflow across a distributed system.

For the purpose of a POC I have created a couple of docker containers each with a worker started with the queue option. I initiated a chain from yet another container. The chain executes successfully across the containers. Now I decided to enable the result_backend to store the results of each task getting executed.

I set the result_backend to postgresql with the schema option. After executing the chain now, I do see the tables created in postgres. However the task_meta table has some columns as null (e.g. the worker, queue.) Where as the task_id and status are correctly populated.

Has anyone faced a similar issue? Any help is appreciated.

celery
1 Answers

I know it has been a long time since you asked the question but I hope it will help others.

By default, Celery does not write all task result attributes to the backend. Your have to configure it by setting result_extended to True as stated here: https://docs.celeryproject.org/en/stable/userguide/configuration.html#result-extended

So if you configure your app from python you will have to do something like:

import celery

app = Celery(
    broker=broker_url,
    backend=postgresq_url,
)

app.conf.update(result_extended=True)
user_96749380
Related