AWS Elastic Beanstalk sqsd error while processing a message

Viewed 562

I have an Elastic Beanstalk Python worker environment. The average job running time is about 20 seconds. Sometimes the following scenario happens,

  • sqsd picks a message from the sqs queue and sends it to the worker.

  • The worker starts processing the message.

  • in few seconds (ranges from 1 to 30 seconds) sqsd gets the following error and parks the message in the Dead letter queue as I configured the retries to 1.

    127.0.0.1 (-) - - [23/Nov/2017:19:48:17 +0000] "POST / HTTP/1.1" 500 527 "-" "aws-sqsd/2.3"

  • The worker continues to process the message and finishes successfully. I have logs to trace that.

That makes the environment in general not healthy.

I have the connection timeout = 60 seconds, Inactivity timeout = 600, Visibility timeout = 600, HTTP connections = 2.

I have the following in the configs as well

option_settings:
  aws:elasticbeanstalk:container:python:
    NumProcesses: 3
    NumThreads: 10

files:
  "/etc/httpd/conf.d/wsgi_custom.conf":
    mode: "000644"
    owner: root
    group: root
    content: |
      WSGIApplicationGroup %{GLOBAL}

Is this because of some memory limit that wsgi puts to every request? That is the only thing that I can think of.

0 Answers
Related