I have 2 t2 instances on AWS, one for beat, one for worker.
Beat (t2.micro): celery -A iso_backend beat -l info --scheduler django
Worker (t2.small): celery -A iso_backend worker -l info --scheduler django
I'm using djang_celery_beat_clockedschedule's table for scheduling a lot of long-running job.
A lot of clocked has_many jobs, so example: at 08:20:00am, Beat must call 100 messages to SQS queue at the same time, the issue is, it call only a fews, sometimes 1/2, sometimes 1/3 number of the jobs
Scheduler: Sending due task...
Actually, this issue is really hard and I cannot figure out the root cause.
Anybody met this before and has any solutions help me?
Thank a lot!