Why is ffmpeg's conversion to YUV420 so poor?

user_1190077 0
Viewed 710

I have been using ffmpeg and other compression tools to compare rate-distortion curves for YUV420-resampled video. In these comparisons, results from ffmpeg are consistently worse, with PSNR values that are 0.5-1.0 dB lower.

I tracked the problem to ffmpeg's conversion between RGB and YUV420. To simplify, let us assume "lossless compression" and therefore consider only RGB -> YUV420 -> RGB. Also, we operate on a single PNG image frame.

# Use some default options.
ffmpeg="ffmpeg -nostdin -hide_banner -v error"

# Obtain a source image.
wget -nv -O original.png https://i.stack.imgur.com/8J1qY.png
size="256x256"

# Compare it with itself to verify that we get an infinite average PSNR.
$ffmpeg -v info -i original.png -i original.png -lavfi psnr -f null - |& grep PSNR
# average:inf

# Convert the image to YUV420, and convert back to RGB.
$ffmpeg -i original.png -pix_fmt yuv420p -f rawvideo -y temp1.yuv420
$ffmpeg -f rawvideo -s $size -pix_fmt yuv420p -i temp1.yuv420 -y result1.png

# Compare it with the original image to measure the PSNR (in dB).
$ffmpeg -v info -i result1.png -i original.png -lavfi psnr -f null - |& grep PSNR
# average:36.894551

Now, as an alternative, we perform the RGB <-> YUV420 chroma resampling manually:

yuv444_to_yuv420="extractplanes=y+u+v[y][u][v];\
  [u]scale=w=iw/2:h=ih/2:flags=area[u];\
  [v]scale=w=iw/2:h=ih/2:flags=area[v];\
  [y][u][v]mergeplanes=0x001020:yuv420p"
yuv420_to_rgb="extractplanes=y+u+v[y][u][v];\
  [u]scale=w=iw*2:h=ih*2:flags=neighbor[u];\
  [v]scale=w=iw*2:h=ih*2:flags=neighbor[v];\
  [y][u][v]mergeplanes=0x001020:yuv444p,format=rgb24"

$ffmpeg -i original.png -pix_fmt yuv444p -f rawvideo - | \
  $ffmpeg -f rawvideo -pix_fmt yuv444p -s $size -i - \
    -lavfi "$yuv444_to_yuv420" -f rawvideo -y temp2.yuv420
$ffmpeg -f rawvideo -pix_fmt yuv420p -s $size -i temp2.yuv420 \
  -lavfi "$yuv420_to_rgb" -y result2.png

# Measure PSNR by comparing with the original image.
$ffmpeg -v info -i result2.png -i original.png -lavfi psnr -f null - |& grep PSNR
# average:37.536444
# This is an improvement of 0.64 dB!

This brings up two questions:

  1. Why doesn't ffmpeg implement a better conversion to/from yuv420p by default?
  2. Is there any simpler way to obtain or express this improved conversion?
ffmpeg
1 Answers

After experimentation, I did manage to find two workarounds:

(1) The argument -sws_flags sets the parameters of all implicitly introduced swscale filters in the filter graph. It appears that these filters are also responsible for the YUV420 conversion. With carefully chosen filter flags, it works:

$ffmpeg -i original.png -sws_flags 'area+accurate_rnd+full_chroma_int' \
  -pix_fmt yuv420p -f rawvideo -y temp1.yuv420

$ffmpeg -f rawvideo -s $size -pix_fmt yuv420p -i temp1.yuv420 \
  -sws_flags 'neighbor+accurate_rnd+full_chroma_int' -y result1.png

$ffmpeg -v info -i result1.png -i original.png -lavfi psnr -f null - |& grep PSNR
# average:37.567842

(2) It is also possible to specify the filtering options as parameters of a scale filter (with default 100% scale) which performs rgb24->yuv420p or yuv420p->rgb24 format conversion depending on the input format requested by the next graph node:

$ffmpeg -v info -i original.png \
  -lavfi 'scale=flags=area+accurate_rnd+full_chroma_int,format=yuv420p' \
  -f rawvideo -y temp1.yuv420

$ffmpeg -v info -f rawvideo -s $size -pix_fmt yuv420p -i temp1.yuv420 \
  -lavfi 'scale=flags=neighbor+accurate_rnd+full_chroma_int' -y result1.png

$ffmpeg -v info -i result1.png -i original.png -lavfi psnr -f null - |& grep PSNR
# average:37.567842

It's unfortunate that this behavior is not the default, but at least there is a semi-convenient way to access it.

user_11900770
Related