Tensorflow getting stuck on prediction when deploying with gunicorn

user_12833553 0
Viewed 216

I've built myself a flask web server, and am deploying it with Gunicorn. It works perfectly well with the default flask app.run(), but this isn't meant for production, so up until now I've had my flask app running on gunicorn.

However, since adding my TensorFlow program to my site, when I go to run a prediction it runs forever, I have timeout set to like 5 minutes. I tested with a print statement before and a print statement after my predict. Second print never shows up.

Any ideas?

gunicorn tensorflow2.0 tensorflow python
1 Answers

If you are running heavy calculations inside the route, you better change your program design.

At your route start the job on some other thread or process and return the status of the process start and ID to query its progress.

Then you call query with polling [or web sockets] your heavy task with your ID, and return its status till the process is done and you can return the results

user_10376130
Related