I have a Flask/Gunicorn endpoint which takes a few seconds to return and gets hit pretty hard. Gunicorn seems to queue up a lot of requests and eventually process them all, but the requests which happen to be added at the back of the queue end up taking a really long time.
The app runs as gunicorn -w 4 -b :8080.
Is there any way to configure Flask/Gunicorn so that it will only keep X requests in the queue?
Probably you've figured it out by now, but since I ran into this, I may as well answer it.
The "request queue" you mention, is called "backlog" and based on the docs you can change it by passing in the --backlog argument to your command, so it would look like:
gunicorn -w 4 --backlog 1024 -b :8080
Keep in mind though, that if the number of requests exceed the number you put there, your clients will start seeing errors.