Is there a multi images to 1 image deep learning method? (pix2pix?)

user_18936409 0
Viewed 49

I'm trying to build a video stabilization deep learning model.
I want to make the model predict how the frame should be stabilized depending on the last 10 frames

I have tried pix2pix, which is image to image, but I didn't get a good result
so, I want the same as pix2pix but multi images to 1 image is there a method or can I do it using pix2pix?

generative-adversarial-network pytorch tensorflow deep-learning python
1 Answers

So, I do not know if you actually need to build this video stabilization using deep learning or if you just want on off-the-shelves solution.

For the on-the-shelves solution, you can look into vidgear that has an awesome stabilisation system built-in: https://abhitronix.github.io/vidgear/latest/gears/stabilizer/overview/

If you want a more advanced solution and architecture, you could take a look at his thread of paper with code: https://paperswithcode.com/task/video-stabilization

Given the current architecture of pix2pix, I do not see how multi-images will provide some stabilisation since, it is just as you said, pix2pix does not consider its previous output nor the flow of images to generate its prediction.

I hope that it helps ^^

user_185797720
Related