Best practice on uploading multiple files (mostly images) to Amazon S3 bucket

Viewed 1015

I have multiple clients like web, mobile web, Android & iOS app as well. Now user is adding some information and along with that also uploading some files.

Now to upload multiple files what would be the good practice:

  • Call my backend rest api to store information in my DB and asynchronously Upload those files (images) from client directly to amazon S3 and then on successful upload call backend rest api to store information such as file location in S3, etc.
  • Make one rest call to backend for storing information and another multipart backend call which in turn makes call to S3 for uploading files and then storing meta info to DB.

Points to be considered:

  • If image or video would be there then some compression would also be required without affecting the quality much. What would be the best way to do it?
  • I believe making it from backend would be a better approach so compression logic would be at one place only instead of scattered at multiple clients. But I think there would be a slight increase in timing as extra rest call would be required.
  • But, User can upload multiple files at once so if we should be making a rest call to backend then is it good to send all those files to backend at once or what would be the better approach, considering mobile apps and mobile web as client as they might have slow internet connection plus too much of network overhead.
  • While uploading from clients to S3, instead of storing credentials I guess there is some another way as well like presigned URL or something so credentials storage would not be required at client side.

Note: It is not a microservice application. Backend is developed in Spring Boot.

Please help to decide on better approach. If anymore details required then please let me know.

1 Answers

Your guess is correct, presigned URL it is a good solution. Presigned URL allows you to upload files (e.g. images/videos) to s3 without the need of AWS credential in the client. The process is to generate a presigned URL that gives you access to upload your file without the need for credentials and with an expiration time. For large files upload you either can zip them in the client and upload them as zip or use multipart upload, this way you won't get a timeout error if the file is too large. I have a nodejs code snippets for both presigned URL and multipart upload, let me know if you want them

Related