I am trying to find the frame in a video that is the most different from its previous and next one. Selecting this frame would give me the best chance of synchronizing two videos coming from different sources.
I am using the psnr filter between the target video and its delayed version, i.e., with ss=1/fps.
Everything seems to work, but something strange happens when I am trying to see which frame (and surrounding frames) was chosen.
I extract the frames as follows:
ffmpeg -ss <target_frame_time> -i <path_to_video> -frames 1 tmp/<frame_name>.png -y
Doing this for the 2 frames before and the 2 frames after the selected frame, I get mse_avg and psnr_avg from the psnr filter as follows:
mse_avg psnr_avg comment
2099.68 14.91 prev2 - prev
3365.92 12.86 prev - keyframe
2716.40 13.79 keyframe - next
2581.58 14.01 next - next2
Instead, when dealing with the video, as well as when extracting single frames in .mp4 format I get:
mse_avg psnr_avg comment
942.57 18.39 prev2 - prev
1611.20 16.06 prev - keyframe
1348.51 16.83 keyframe - next
1208.34 17.31 next - next2
I thought that saving frames in lossless image format would preserve their visual characteristics, thus providing the same performance. Why does changing the frame output to PNG format yield different MSE?