Video conversion with ffmpeg to target Android and iOS mobile devices

user_1735653 0
Viewed 7032

I'm building a react native app for both Android and IOS, the back-end API is written with NodeJS.

Users may upload video from their phones, once uploaded the user and their friends will be able to view the video - so the videos need to be stored in a format which is playable on both Android & IOS.

My question relates to the conversion of video, uploaded by the user. I developed a similar app a couple of years ago; I used the repo node-fluent-ffmpeg which provides a nice API to interact with FFmpeg.

In the previous project (which was a web app), I converted the uploaded videos into two files, one .mp4 and one .webm - if a user uploaded an mp4, then I would skip the mp4 step, likewise if they uploaded a .webm.

This was kind of slow. Now I've come across the same requirement years later, after some research I think I was wrong to convert the videos to the last project.

I've read that I can simply use FFmpeg to change the container format of the videos, which is a much faster process than converting them from scratch.

The video conversion code I used last time went something along the lines of: 

var convertVideo = function (source, format, output, success, failure, progress) {

    var converter = ffmpeg(source);

    var audioCodec = "libvorbis";

    if (format.indexOf("mp4") != -1) {
        audioCodec = "aac";
    }

    converter.format(format)
        .withVideoBitrate(1024)
        .withAudioCodec(audioCodec)
        .on('end', success)
        .on('progress', progress)
        .on('error', failure);

    converter.save(output);
};

Usage:

Convert to mp4:

convertVideo("PATH_TO_VIDEO", "mp4", "foo.mp4", () => {console.log("success");});

Convert to webm:

convertVideo("PATH_TO_VIDEO", "webm", "foo.webm", () => {console.log("success");});

Can anyone point out a code smell here regarding the performance of this operation? Is this code doing a lot more than it should achieve cross-platform compatibility between IOS and Android?

Might be worth mentioning that support for older OS versions is not such a big deal in this project.

node.js ffmpeg video android ios
3 Answers

Thats easy, you just go for the same that the big vendors like youtube do, which is currently mp4/h.264/aac

As you target Android or better "any" OS or device, to get out the max. you have additionally to take care about the detailed settings of the codecs. Hardware decoders that are built in to devices tend to be pretty picky about what they want to decode.

Youtube defines the video codec settings this way

for the h264:

  • Progressive scan (no interlacing)
  • High Profile
  • 2 consecutive B frames
  • Closed GOP. GOP of half the frame rate.
  • CABAC
  • Variable bitrate. No bitrate limit required, though we offer recommended bit rates below for reference
  • Chroma subsampling: 4:2:0

The only thing i can add: actually you should care about the bitrate, if you go too high (e.g. more than 15MBit/s for Full-HD video), hardware decoders might cause troubles.

For the AAC:

  • Channels: Stereo or Stereo + 5.1
  • Sample rate 96khz or 48khz

Container: MP4

  • No Edit Lists (or the video might not get processed correctly)
  • moov atom at the front of the file (Fast Start)

Furthermore, h264 is definitely the codec that has been worked on most which is why it can be encoded much faster than VP8/9 and co. So in my opinion there is no good reason (besides "ethic" ones) to go for h264 ONLY - not to mention that "x264" is even faster than h264 and should guarantee the same compatibility

Conclusio:

Can anyone point out a code smell here regarding performance of this operation? Is this code doing a lot more than it should to achieve cross platform compatibility between IOS and Android?

Unfortunately no, it all depends on what kind of videos you get as input. Your code does definitely not too much, but far to less in terms of "try to save computing time". You could check if the input video matches all the listed parameters before you send it to encoding but to be honest, from experience most compatible results you will get is when you re-encode all your inputs to the settings above.

user_87291030
Related