Celery/Kombu worker under Docker worker won't pick up messages from SQS

Question

I'm enqueuing jobs to AWS SQS from a local env and I'm able to consume them successfully from the same local env but when I try to pack the worker into a docker instance and run it, it will not pick up the jobs. As far as I can tell, it is hitting the right queues, there are no errors, but it just won't consume the jobs.

Python env

python --version
Python 3.9.6

pip list
amqp             5.0.7
billiard         3.6.4.0
boto3            1.20.24
botocore         1.23.24
celery           5.2.1
click            8.0.3
click-didyoumean 0.3.0
click-plugins    1.1.1
click-repl       0.2.0
jmespath         0.10.0
kombu            5.2.2
pip              21.1.3
prompt-toolkit   3.0.24
pycurl           7.44.1
python-dateutil  2.8.2
pytz             2021.3
s3transfer       0.5.0
setuptools       56.0.0
six              1.16.0
urllib3          1.26.7
vine             5.0.0
wcwidth          0.2.5

requirements.txt

celery[sqs]

local environment

export AWS_ACCESS_KEY_ID="AKIA*********"
export AWS_SECRET_ACCESS_KEY="**********"

my 'task code

from celery import Celery
import pprint

app = Celery('tasks')
app.conf.broker_transport = 'sqs'
app.conf.broker_transport_options = {
    'region': 'ap-southeast-2'
}
app.conf.result_backend = 'rpc://'

print("----------------------------------------------")
pprint.pprint(app.conf)
print("----------------------------------------------")

@app.task
def add(x, y):
    return x + y

consuming messages from local

celery -A tasks worker --loglevel=info

Dockerfile

FROM python:3.6.15-alpine3.13

COPY ./app /app

WORKDIR /app

CMD ash -c "apk add --no-cache build-base libcurl curl-dev; pip3 install -r requirements.txt; celery -A tasks worker --loglevel=debug"

Log of consuming messages running local (mac)

https://pastebin.com/uudPyJwu

Log of (not) consuming messages running from docker

https://pastebin.com/giMuahKJ

AWS console showing one message successfully consumed using local and a second message sat there doing nothing

Screen Shot 2021-12-19 at 11 48 02 am

user_4465537 · Answer

I have an exp using celery with queues in docker and I have met the issue you have. I would first start from fixing your Dockerfile

FROM python:3.6.15-alpine3.13
COPY ./app /app
WORKDIR /app
# FIRST install os packages.
RUN apk add --no-cache build-base libcurl curl-dev
# Then perform pip install for modules.
RUN pip3 install -r requirements.txt
# have a script that runs your application
ENTRYPOINT [ "/app/entry.sh" ]

entry.sh inside app folder should looks like this:

#!/bin/bash

# if any of the commands in your code fails for any reason, the entire script fails
set -o errexit
# fail exit if one of your pipe command fails
set -o pipefail
# exits if any of your variables is not set
set -o nounset

cd /app
celery -A tasks worker --loglevel=debug
python3 app.py

Even though you have started a celery instance, you still need to run your application code (the one you specified at my 'task code)

Try my suggestions and feel free to comment if you still got any issues.