Celery task to process data from database Postgres and django

user_9051246 0
Viewed 1217

I'm trying to use celery and celery beat to run a scheduled task to process data from the database, but when I try to run the task I get this error "django.db.utils.OperationalError: FATAL: role "tanaka" does not exist". The code for the scheduled task is shown below

settings.py

CELERY_BEAT_SCHEDULE = {
    'task-number-one': {
        'task': 'loans.tasks.update_loan_book',
        'schedule': 60,
    },
}

tasks.py

@shared_task
def update_loan_book():
    tenants = Tenant.objects.all()
    for tenant in tenants:
        #logic to update tenant object

The code works when I run the task using the "celery -A proj worker -l info -B" command but does not work when I daemonize celery and celery beat. Config files for celery and celery beat are shown below. I am using supervisord.

[program:projworker]
command=/home/tanaka/microfinance/bin/celery -A cloud_based_microfinance worker -l info
directory=/home/tanaka/Repositories/microfinance_project
user=tanaka
numprocs=1
stdout_logfile=/var/log/celery/proj_worker.log
stderr_logfile=/var/log/celery/proj_worker.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600
killasgroup=truepriority=998

[program:projbeat]
command=/home/tanaka/microfinance/bin/celery -A cloud_based_microfinance beat -l info
directory=/home/tanaka/Repositories/microfinance_project
user=tanaka
numprocs=1
stdout_logfile=/var/log/celery/proj_beat.log
stderr_logfile=/var/log/celery/proj_beat.log
autostart=true
autorestart=true
startsecs=10
priority=999

When I try to run the task as a daemon I get "django.db.utils.OperationalError: FATAL: role "tanaka" does not exist" in the proj_worker.log file.

celery supervisord django
1 Answers

Solved the problem, if your task is going to access the database your first need a db connection

from django.db import connection

@shared_task
def update_loan_book():
  with connection.cursor() as cursor:
    tenants = Tenant.objects.all()
    for tenant in tenants:
      #logic to update tenant object

Celery and django are two different processes therefore you have to import db connection from django for celery to access the db. If you are not using django ORM this is a helpful resource

user_90512460
Related