how many uvicorn workers do I have to have in production?

Viewed 2902

My Environment

  • FastAPI
  • Gunicorn & Uvicorn Worker
  • AWS EC2 c5.2xlarge (8 vCPU)

Document

https://fastapi.tiangolo.com/deployment/server-workers/

Question

Currently I'm using 24 Uvicorn workers in production server. (c5.2xlarge)

gunicorn main:app --workers 24 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:80

I've learn that one process runs on one core. Therefore If i have 8 processes, I can make use of whole cores (c5.2xlarge's vCpu == 8)

I'm curious that in this situation, Is there any performance benefit if I got more processes than 8?

2 Answers

Number of recommended workers is 2 x number_of_cores +1

You can read more about it at https://docs.gunicorn.org/en/stable/design.html#:~:text=Gunicorn%20should%20only%20need%204,workers%20to%20start%20off%20with.

In your case with 8 CPU cores, you should be using 17 worker threads.

Additional thoughts on async systems:

The two times core is not a scientific figure as says in the article. But the idea is that one thread can do I/O and another CPU processing at the same time. This makes maximum use of simultaneous threads. Even with async systems, conceptually this works and should give you maximum efficiency.

In general the best practice is:

number_of_workers = number_of_cores x 2 + 1

or more precisely:

number_of_workers = number_of_cores x num_of_threads_per_core + 1

The reason for it is CPU hyperthreading, which allows each core to run multiple concurrent threads. The number of concurrent threads is decided by the chip designers.

Two concurrent threads per CPU core are common, but some processors can support more than two.

vCPU that is mentioned for AWS ec2 resource is already the hyperthreaded amount of processing units you have on the machine (num_of_cores x num_of_threads_per_core). Not to be confused with number_of_cores available on that machine.

So, in your case, c5.2xlarge has 8 vCPUs, meaning you have 8 available concurrent workers.

Related