How to implement Celery using Flask application factory pattern

Viewed 4459

I am having issues with implementing celery with python flask application factory app

I have intend creating an instance of the Celery app from the app init file as below:

from celery import Celery
celery = Celery('myapp', broker='redis://localhost:6379/0', backend='redis://localhost:6379/0')

I can't use Celery from other blueprint when called.

4 Answers
def init_celery(app):
    celery = Celery()
    celery.conf.broker_url = app.config['CELERY_BROKER_URL']
    celery.conf.result_backend = app.config['CELERY_RESULT_BACKEND']
    celery.conf.update(app.config)

    class ContextTask(celery.Task):
        """Make celery tasks work with Flask app context"""
        def __call__(self, *args, **kwargs):
            with app.app_context():
                return self.run(*args, **kwargs)

    celery.Task = ContextTask
    return celery

Initizialize celery when create_app:

init_celery(app)

Find how celery is implemented in this Flask cookiecutter

The answer by Joost Döbken may work but it seems a bit more complicated than it has to be.

I found simpler solution by Miguel Grinberg that works great for me:

from celery import Celery
from config import config, Config

celery = Celery(__name__, broker=Config.CELERY_BROKER_URL)

def create_app(config_name):
    # ...
    celery.conf.update(app.config)
    # ...
    return app

https://blog.miguelgrinberg.com/post/celery-and-the-flask-application-factory-pattern

Answer from @API was correct... Also add the following to your celery configuration... This will help prevent the unending retry caused by celery when broker is down or could not be reached..

broker_transport_options  = {
'max_retries': 3,
'interval_start': 0,
'interval_step': 0.2,
'interval_max': 0.5,
}

Version of celery used is == 4.3 as at when this question was answered.

The flask-execute plugin supports the application factory pattern out of the box:

from flask import Flask
from flask_execute import Celery

celery = Celery()
app = Flask(__name__)
celery.init_app(app)
Related