I am using Celery for excuting various tasks from django. The celery is managed using Supervisor. Rabbitmq is used the broker for celery . Celery and django are inside a single Docker container and broker in another. The entire application is working Docker-compose running in RHEL 7.9.
Problem
- celery is continuously restarting after every 2min in supervisor.
- no log event about the same is captured.
Code
Supervisor config
[program:django-celery]
command=/usr/local/bin/celery -A CELERY_APP worker --loglevel=info --logfile=/var/log/supervisor/celery.log
directory=/app
user=root
numprocs=1
stdout_logfile=/var/log/supervisor/celery_out.log
stderr_logfile=/var/log/supervisor/celery_err.log
autostart=true
autorestart=true
startsecs=10
stopwaitsecs = 600
killasgroup=true
priority=998
environment=DJANGO_SETTINGS_MODULE='CELERY_APP.settings',CONFIG_TYPE='PROD',LD_LIBRARY_PATH='/usr/lib/oracle/18.3/client64/lib/'
[program:django-celerybeat]
command=/usr/local/bin/celery -A CELERY_APP beat --loglevel=info --logfile=/var/log/supervisor/celery_beat.log
directory=/app
user=root
numprocs=1
stdout_logfile=/var/log/beat_out.log
stderr_logfile=/var/log/supervisor/beat_err.log
autostart=true
autorestart=true
startsecs=10
priority=999
environment=DJANGO_SETTINGS_MODULE='CELERY_APP.settings',CONFIG_TYPE='PROD',LD_LIBRARY_PATH='/usr/lib/oracle/18.3/client64/lib/'
Dockerfile
FROM python:3.8
ENV PYTHONUNBUFFERED=1
RUN mkdir /app
WORKDIR /app
EXPOSE 8000 5672 15672
COPY requirements.txt /app/
RUN pip install -r requirements.txt
RUN apt-get update
RUN apt-get install -y wget net-tools curl alien unzip telnet supervisor zlib1g
RUN apt-get update && apt-get install -y libaio1
RUN wget http://yum.oracle.com/repo/OracleLinux/OL7/oracle/instantclient/x86_64/getPackage/oracle-instantclient18.3-basic-18.3.0.0.0-3.x86_64.rpm
RUN alien -i oracle-instantclient18.3-basic-18.3.0.0.0-3.x86_64.rpm
ENV LD_LIBRARY_PATH=/usr/lib/oracle/18.3/client64/lib/${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}
COPY . /app/
RUN mkdir -p /app/TEMP/PIXL
RUN mkdir -p /app/TEMP/RUNNER
RUN mkdir -p /app/TEMP/DUMP
COPY docker/django/start /start
RUN chmod +x /start
COPY docker/django/supervisor_app.conf /etc/supervisor/conf.d/supervisor_app.conf
COPY docker/django/supervisord.conf /etc/supervisor/supervisord.conf
celery.py
import os
from celery import Celery
from celery.schedules import crontab
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'CELERY_APP.settings')
app = Celery("CELERY_APP")
app.config_from_object('django.conf:settings', namespace='CELERY')
app.conf.beat_schedule = {
'zip_and_send': {
'task': 'asyncManagment.tasks.create_zip_file_for_push',
'schedule': crontab(minute='*/30'),
},
'parse_response': {
'task': 'asyncManagment.tasks.response_manager',
'schedule': crontab(minute=0,hour='*/3'), #crontab(minute='*',hour='8-19')
},
'zip_and_resend': {
'task': 'asyncManagment.tasks.create_zip_file_for_repush',
'schedule': crontab(minute=6,hour='14'),
},
'value_check':{
'task':'asyncManagment.views.db_check',
'schedule': crontab(minute=6,hour='12'),
},
}
app.autodiscover_tasks()
Could someone please tell me the possible cause and solution? If I've missed out anything, over- or under-emphasized a specific point, let me know in the comments. Thank you so much in advance for your time.