We've a system that spews out 4-channel png images frame-by-frame (we control the output format of these images as well, so we can use something else as long as it supports transparency). Right now, we're waiting for all the images and then encoding them with ffmpeg into a webm video file with vp8 (libvpx encoder). But we now want to pipeline these images to FFmpeg to encode into the WebM video simultaneously as the images are being spewed out so that we don't wait for ffmpeg to encode all the images afterwards.
This is the current command, in python syntax:
['/usr/bin/ffmpeg', '-hide_banner', '-y', '-loglevel', 'info', '-f', 'rawvideo', '-pix_fmt', 'bgra', '-s', '1573x900', '-framerate', '30', '-i', '-', '-i', 'audio.wav', '-c:v', 'libvpx', '-b:v', '0', '-crf', '30', '-tile-columns', '2', '-quality', 'good', '-speed', '4', '-threads', '16', '-auto-alt-ref', '0', '-g', '300000', '-map', '0:v:0', '-map', '1:a:0', '-shortest', 'video.webm']
# for ease of read:
# /usr/bin/ffmpeg -hide_banner -y -loglevel info -f rawvideo -pix_fmt bgra -s 1573x900 -framerate 30 -i - -i audio.wav -c:v libvpx -b:v 0 -crf 30 -tile-columns 2 -quality good -speed 4 -threads 16 -auto-alt-ref 0 -g 300000 -map 0:v:0 -map 1:a:0 -shortest video.webm
proc = subprocess.Popen(args, stdin=subprocess.PIPE)
Here is a sample example of passing the image to FFMPEG proc stdin as:
# wait for the next frame to get ready
for frame_path in frame_path_list:
while not os.path.exists(frame_path):
time.sleep(0.25)
frame = cv2.imread(frame_path, cv2.IMREAD_UNCHANGED)
# put the frame in stdin so that it gets ready
proc.stdin.write(frame.astype(np.uint8).tobytes())
The current speed of this process is ~0.135x which is a huge bottleneck for us. Earlier when we were taking input as -pattern_type glob -i images/*.png we were getting around 1x-1.2x for this on a single core. So, our conclusion is that we're getting bottlenecked by stdin and hence are looking for ways to pass input through multiple sources or somehow help ffmpeg to parallelize this effort - a few options that we're thinking of:
- Somehow feed it to different pipes and make ffmpeg read from them.
- Append a new image to ffmpeg without re-encoding the whole video, but we didn't find a way to do this with giving input images directly.
But we haven't been able to get either of these working, open to any other solutions as well. Will really appreciate the help on this. Thanks!