How to handle long time processing request

user_10246394 0
Viewed 413

I have a spring boot rest API deployed on AWS Elastic Beanstalk and I am trying to upload pictures through it.

This is what I did : Upload a zip file through a file input from the browser, get the zip file on the server, go through all the files and upload each one on AWS S3.

It works fine but I ran into a problem: When I try to upload lots of pictures, I get an HTTP error (504 Gateway Timeout). I found out this is because the server takes too much time to respond, and I am trying to figure how to set a higher timeout for the requests (didn't find yet). But in the mean time I am asking myself if it is the best solution.

Wouldn't it be better to end the request directly after receiving the zip file, make the uploads to S3 and after that notify the user that the uploads are done ? Is there even a way to do that ? Is there a good practice for this ? (operation that takes lots of time to process).

I know how to do the process asynchronously but I would really like to know how to notify the user after it completes.

amazon-web-services rest api spring
1 Answers

Wouldn't it be better to end the request directly after receiving the zip file, make the uploads to S3 and after that notify the user that the uploads are done ?

Yes, asynchronous processing of the uploaded images in the zip file would be better.

Is there even a way to do that ? Is there a good practice for this ? (operation that takes lots of time to process).

Yes there is a better way. To keep everything within EB, you could look at Elastic Beanstalk worker environment. The worker environment is ideal for processing your images.

In this solution, your web based environment would store the images uploaded in S3 and submit it names along with other identifying information to an SQS queue. The queue is an entry point for the worker environment.

Your workers would process the images from the queue independently from the web environment. In the meantime, the web environment would have to check for the results and notify your users once the images get processed.

The EB also supports linking different environments. Thus you could establish a link between web and worker environments for easier integration.

user_2488230
Related