Celery beat process allocating large amount of memory at startup

user_6479734 0
Viewed 650

I operate a Django 1.9 website on Heroku, with Celery 3.1.23. RabbitMQ is used as a broker.

After restarting the beat worker, the memory usage is always around 497Mb. This results in frequent Error R14 (Memory quota exceeded) as it quickly reaches the 512Mb limit.

How can I analyze what is in the memory at startup? I.e. how can I get a detail of what is in the memory when restarted?

Here is a detail of memory consumption obtained with the beta Heroku log-runtime-metrics:

heroku/beat.1:
source=beat.1 dyno=heroku.52346831.1ea92181-ab6d-461c-90fa-61fa8fef2c18
sample#memory_total=497.66MB
sample#memory_rss=443.91MB
sample#memory_cache=20.43MB
sample#memory_swap=33.33MB
sample#memory_pgpgin=282965pages
sample#memory_pgpgout=164606pages
sample#memory_quota=512.00MB
celery rabbitmq heroku memory django
1 Answers

I had the same problem. Searching around, I followed How many CPU cores has a heroku dyno? and Celery immediately exceeds memory on Heroku.

So I typed:

heroku run grep -c processor /proc/cpuinfo -a <app_name>

It returned 8. So I added --concurrency=4 to my Procfile line:

worker: celery -A <app> worker -l info -O fair --without-gossip --without-mingle --without-heartbeat --concurrency=4

And memory usage seemed to be divided by almost 2:

Screenshot of memory usage of worker Dyno on Heroku after the explained changes

user_7079840
Related