Flask EB 504 Gateway Time-out

user_14017915 0
Viewed 2421

I am new to flask and recently created a flask application where user uploads a xlsx file and gets back another xlsx file after processing.

I have hosted this flask application using AWS EB free tier. Everything works fine if the processing time is less than 30 secs. But i have cases where the processing time is large and unknown(15-20 minutes). When the computation is large i get this 504 error.

I would like to know what would be an ideal solution to handle these kinds of requests. I have seen people giving suggestions on ngnix and gunicorn configurations but i don't have any idea about them.

It would be great if anyone can take time and give an idea of what can be done.

gunicorn flask amazon-web-services nginx python
2 Answers

The HyperText Transfer Protocol (HTTP) 504 Gateway Timeout server error response code indicates that the server, while acting as a gateway or proxy, did not get a response in time from the upstream server that it needed in order to complete the request. The request expired before the response was given - MDN

You should consider doing time intensive processing as a background task. (related Guide)

If you need two app structures for your apps you can unify (in modules) them into one by taking advantage of Flask blueprints.

user_126489820

I would suggest that you look at performing some rearchitecting for this process which will help to provide a better solution for your users.

You should look at using additional technologies to break apart the processing and return of the new xlsx file from the original HTTP request. By decoupling these apart your end user will receive an improved experience with less chance of a failure along the way (at the moment everything is relying on a HTTP request, if it fails this whole process fails).

Your HTTP request will need to accept the xslx file, after this it should store the file and queue the processing of this action. If you stored the file in S3, you could then make use of S3 events to trigger a Lambda to handle this processing. If you're performing a sequence of events it might be more appropriate to break these steps up as individual functions that could be triggered by a Step Function.

Your application would need to understand that the action is queued, so you would need to decide how you would like to expose this information to the user. You might choose to store the status of processing to say it is PENDING until your step function marks as complete, or perhap you might choose to email a completed file to the user (using a presigned URL).

user_134609330
Related