Nodejs upload and process video

user_14793553 0
Viewed 211

I'm looking for best approach for my application. I have video upload functionality. Front-end will send upload/video request with attached video file, then my Back-End will handle this request, will reduce the size and quality of the video (using fluent-ffmpeg ), then will create thumbnail image, based on the first frame of the video, then will upload video and his thumbnail image to the AWS S3 bucket, and in finally will return the compressed video and thumbnail to the front-end. The problem that i have, all of those (back-end) tasks for compressing, creating thumbnail and uploading are much time consuming and sometimes (depends on the video size and duration) my nginx server will return 504 Gateway Time-out, which is normal. The question is:

How to handle this case. Should i use web sockets to notify the front-end for progression for processing the video, or i don't need to wait until all of those actions is completed. My goal is to have functionality, where i can upload a video and show some progress bar for video processing and the user can be able to "play" with the application, not to be required to waiting until video is processed successfully

aws-sdk video-processing node.js file-upload stream
1 Answers

Seems like this is an architectural problem. Here is one of the solution that I prefer.

  1. Use queue and store progress in some key value db. You may be unfamiliar with queue so I would recommend you to check some queue related tutorials. As you are using amazon, sqs might be interesting to you. In Rails you can check sidekiq. Laravel has laravel horizon.

  2. While each queue is progress design app so it can report it. Like 50% 60% etc.

  3. Process Thumbnails etc on queue too.

And if you want to scale you can simply increase the number of queue. I think that's how other also handle it.

user_53364630
Related