Protocol Error when using celery worker with Flask

Viewed 428

I am running remote celery worker with Flask. The configuration in flask is redis is used for backend and rabbitmq as message broker.

Flask is running with multiple threads, and the celery get() is used at two places.

@app.routes("/route1")
def method1():
    result1_obj = remote_method_1.apply_async()
    result1 = result1_obj.get()

@app.routes("/route2")
def method2():
    result2_obj = remote_method_2.apply_async()
    result2 = result2_obj.get()

So whenever the get() is called by two different threads at the same time under method1() and method2().

Error logs are as follows:-

Traceback (most recent call last):
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\flask\app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\flask\app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\flask_restplus\api.py", line 325, in wrapper
    resp = resource(*args, **kwargs)
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\flask\views.py", line 89, in view
    return self.dispatch_request(*args, **kwargs)
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\flask_restplus\resource.py", line 44, in dispatch_request
    resp = meth(*args, **kwargs)
  File "C:\Users\venugopal.venkatesh\OneDrive - Veoneer\Documents\sprint35\fdt_report_gen\src\application\routes.py", line 89, in wrap
    return f(*args, **kwargs)
  File "C:\Users\venugopal.venkatesh\OneDrive - Veoneer\Documents\sprint35\fdt_report_gen\src\db\__init__.py", line 281, in wrap
    return f(*args, **kwargs)
  File "C:\Users\venugopal.venkatesh\OneDrive - Veoneer\Documents\sprint35\fdt_report_gen\src\doors\routes.py", line 141, in post
    abort_res = abort_obj.get()
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\celery\result.py", line 237, in get
    on_message=on_message,
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\celery\backends\asynchronous.py", line 200, in wait_for_pending
    for _ in self._wait_for_pending(result, **kwargs):
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\celery\backends\asynchronous.py", line 268, in _wait_for_pending
    on_interval=on_interval):
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\celery\backends\asynchronous.py", line 55, in drain_events_until
    yield self.wait_for(p, wait, timeout=interval)
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\celery\backends\asynchronous.py", line 64, in wait_for
    wait(timeout=timeout)
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\celery\backends\redis.py", line 160, in drain_events
    message = self._pubsub.get_message(timeout=timeout)
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\redis\client.py", line 3617, in get_message
    response = self.parse_response(block=False, timeout=timeout)
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\redis\client.py", line 3505, in parse_response
    response = self._execute(conn, conn.read_response)
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\redis\client.py", line 3479, in _execute
    return command(*args, **kwargs)
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\redis\connection.py", line 739, in read_response
    response = self._parser.read_response()
  File "C:\Users\venugopal.venkatesh\.virtualenvs\fdt_report_gen-BnOKsvM9\lib\site-packages\redis\connection.py", line 331, in read_response
    raise InvalidResponse("Protocol Error: %r" % raw)
redis.exceptions.InvalidResponse: Protocol Error:

Any pointers on how to solve this error?

1 Answers

This is a bug in Celery library https://github.com/celery/celery/pull/6416

backend object is not thread safe nowadays!

You need to run a Flask app without threads, for example, uwsgi --processes 12 --threads 1 --http 0.0.0.0:5000 --module server:app

Related