I am currently working on a microservices architecture on my project and all my services app1, app2, ... are using celery and celery beat.
Is it possible to link all my apps to one celery instance? If yes could you give me a docker-compose example? Or should I create a celery instance for every app or service I have in my project?
Each instance of celery is managing the tasks of one app, also it is needed to specify its own queue to avoid conflicts. Another solution is to make a mixed worker with multiple queues if many needed in a single app.
worker:
build:
context: .
dockerfile: Dockerfile
command: ['celery', '--app=app.celery', 'worker', '--loglevel=INFO',
'--queues=monitor']
environment:
- BROKER_URL=amqp://admin:pass@rabbitmq:5672//
- RESULT_BACKEND=redis://redis:6379/0
- C_FORCE_ROOT=true
volumes:
- /app/:/app/
depends_on:
- rabbitmq
- redis