We have a celery worker and a beat scheduler running in production, in a dedicated Kubernetes POD. We used gevent and run celery with concurrency set to 200.
celery -A app worker -l info --pool=gevent --concurrency=200
Recently, we needed to have a periodic task to be triggered every 5 seconds. But when there is another celery task in progress (especially the ones that interact with the database), we noticed the task is delayed a lot. Checking the beat-scheduler, the tasks are sent every 5 seconds without problems but they are waiting in the broker. We hoped this periodic task was not affected by other tasks because we used concurrency.
Please see screenshot of logs
We need to make sure this task runs every 5 seconds regardless of other tasks.