When i create a long term tasks(1-2 days) some of them are run successfully, but some raises Connection Reset by peer error(104). For example tasks run at 11:30 failed(but some are successful), but tasks run at 12:00 or before 11:30 are successful. settings.py
DATABASES['default']['CONN_MAX_AGE'] = 0
if REDIS_PRIMARY_URL:
CELERY_BROKER_URL = os.environ.get("CELERY_BROKER_URL", REDIS_PRIMARY_URL)
else:
CELERY_BROKER_URL = _env_get_required("CELERY_BROKER_URL")
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-result_backend
CELERY_RESULT_BACKEND = CELERY_BROKER_URL
CELERY_WORKER_MAX_TASKS_PER_CHILD=500
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-accept_content
CELERY_ACCEPT_CONTENT = ["json"]
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-task_serializer
CELERY_TASK_SERIALIZER = "json"
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-result_serializer
CELERY_RESULT_SERIALIZER = "json"
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#beat-scheduler
CELERY_BEAT_SCHEDULER = "django_celery_beat.schedulers:DatabaseScheduler"
CELERY_ACKS_LATE = True
How I run scheduler and worker on aws using containers
["celery","--app=app.celery_app","beat","--loglevel=INFO"] - start scheduler
["celery","--app=app.celery_app","worker","--loglevel=DEBUG","-n worker.%%h","--without-gossip","--without-mingle","--without-heartbeat","-Ofair","--pool=solo"] - start worker
Celery version 5.2