Videos which don't play in Firefox: How to detect and how to fix?

Question

I have, as test cases, two mp4 videos: they seem to be encoded the same, but one works in Firefox and one doesn't:

bad.mp4 (doesn't work) view download

good.mp4 (does work) view download

(NOTE - I thought that the "view" links above would open them using the browser's built-in player, but it looks like they use Dropbox's video player, which DOES work in firefox. So, to see it in firefox, download it and then drag it into Firefox or right click and choose Firefox from "open with" - thanks!)

They both play in Chrome.

There is a long discussion on the Mozilla bug tracking site about a related issue: https://bugzilla.mozilla.org/show_bug.cgi?id=1130450

So, I need to answer three questions:

1) What is the problem with bad.mp4?

2) How can I detect, programmatically (ie at the linux command line) which videos, of a library of videos, are going to have this problem?

3) How can I fix it, programmatically (eg with FFMPEG).

One thing I noticed was that in the good one, the video is on stream 0 and the audio stream 1, while on the bad one it's the other way round. I tried swapping the streams but it didn't seem to help.

EDIT: re the Baseline profile

It was suggested that FF might not like Baseline profile videos. I was able to identify some baseline profile videos that work and some that don't, so I think that's not the key problem. Here are a couple more samples:

Baseline_bad.mp4 (doesn't work) view download

Baseline_good.mp4 (does work) view download

user_12256907 · Answer

First of all sorting mp4 tracks has nothing to do with that. Second, on Win10 with Firefox version 52.5.3 your bad.mp4 works just fine. However on MacOS using Firefox 69.03 bad.mp4 also plays but with a lot of artefacts.

The H.264 bitstream seems to be ok. I demultiplexed the bad mp4 with MP4Box MP4Box -raw 2 bad.mp4 which gives bad_track2.h264 and I decoded it with the reference software. Decoded just fine, without any problems, so the bitstream seems to be not a problem.

So it has to be the packaging into mp4. And indeed, if I package the bitstream back again using MP4Box -add bad_track2.h264 repackage.mp4, it plays just great. So this is a good news, since you don't have to re-encode (transcode) your videos, which will save you a lot of time. Of course transcoding also solves it ffmpeg -i bad.mp4 transcode.mp4 in case you don't want to do it with MP4Box.

To answer 2 and 3. not 100 % sure if this is the same for all of your videos but the difference between good.mp4 and bad.mp4 is the compatible_brands which you can get with ffprobe. you can write a script which finds the videos with the same brand as in bad.mp4 and re-packages each of them using ffprobe, MP4Box.

I hope this helps.

Edit

To address the issue raised in @llogan's answer, regarding the Baseline AVC profile. IMHO this is very unlikely that the Baseline Profile is not supported, however can not be 100% sure about it since it may depend on the platform on which the Firefox is running.

From Mozilla's Web Video codec guide:

1: Firefox support for AVC is dependent upon the operating system's built-in or preinstalled codecs for AVC and its container in order to avoid patent concerns.

user_1109017 · Answer

What is the problem with `bad.mp4`?

It's not the stream order because you can switch the order in good.mp4 and it still plays: ffmpeg -i good.mp4 -map 0:a -map 0:v -c copy switched.mp4

AUD NAL units

bad.mp4 may contain AUD NAL units Firefox doesn't like.

Baseline profile

Perhaps whatever decoder your Firefox is using doesn't like the Baseline H.264 profile. This is not the same as the far more common Constrained Baseline profile which is what good.mp4 is.

I'm not sure what version of Firefox you're using and what OS you are on so it is a guess.

How can I detect this problem?

Use ffprobe to determine the profile:

ffprobe -v error -select_streams v -of csv=p=0 -show_entries stream=profile input.mp4

If the issue is due to the AUD NAL units, and all of the Baseline videos have this issue, then this can still be useful. I didn't take the time to investigate further regarding the AUD NAL units so maybe it's a red herring.

How can I fix it?

If the AUD NAL units are the issue

Remux using the h264_metadata bitstream filter:

ffmpeg -i input.mp4 -bsf:v h264_metadata=aud=remove -map 0 -c copy -movflags +faststart output.mp4

This is fast because it avoids re-encoding.
You'll need FFmpeg 3.4 or later to use this.
See note below regarding -movflags +faststart.

If Baseline profile is the issue

You'll have to re-encode the video to change the profile:

ffmpeg -i input.mp4 -c:v libx264 -c:a copy -movflags +faststart output.mp4

See FFmpeg Wiki: H.264 for more encoding options.
I added -movflags +faststart because it seems like you'll be presenting these videos via progressive download playback. This option will allow the video to begin playing while it is still downloading.