Serving pytorch model with django and uwsgi, how can the system support multi requests concurrency?

Viewed 411

If I street-test the API with 10 concurrency requests:

  • using django built-in server, python3 manage.py runserver, the average response time is about 10 times longer than 1 concurrency request
  • using uwsgi to start 10 progress, the average response time is about 5 times longer than 1 concurrency request

How can I handle multi requests in instant response time?

0 Answers
Related