I've developed a video joining service. Here is how it works in brief.
Step 1: Users upload their video (different video file types) from desktop/mobile. These videos are processed on the server, a logo is added, and converted to .mp4 format.
Step 2: All received mp4 files are then joined, a background music is added and saved.
Everything is working as expected. The only problem is, when video files are joined. The final output video has audio out of sync in some parts. In step 2, I get "Non-monotonous DTS in output stream 0:1;" error message.
Following are step 1 and step 2 codes.
Step 1:
ffmpeg -y -i inputVideo -i inputImage -r 30000/1001 -video_track_timescale 30k -filter_complex "[0]scale=854:480:force_original_aspect_ratio=decrease,pad=854:480:(ow-iw)/2:(oh-ih)/2:black,setsar=1/1,setdar=16/9[v0];[1:v]format=argb,colorchannelmixer=aa=1.0,scale=71:72[overl1],[v0][overl1]overlay=297:285[v1]" -map "[v1]" -map 0:a -b:v 250k -maxrate 250k -bufsize 500k -crf 5 -preset slower -vbr 4 -movflags +faststart -b:a 96k -async 1 outputVideo.mp4
Step 2:
ffmpeg -safe 0 -f concat -segment_time_metadata 1 -i videoList.txt -stream_loop -1 -i backgroundMusic.wav -filter_complex '[0:a]aformat=sample_fmts=fltp:sample_rates=44100:channel_layouts=stereo,volume=1.5[a0];[1:a]aformat=sample_fmts=fltp:sample_rates=44100:channel_layouts=stereo,volume=0.03[a1];[a0][a1]amerge=inputs=2[a]' -map 0:v -map '[a]' -c:v copy -ac 2 finalVideo.mp4
I'm not expert with ffmpeg, and not sure, what's missing. can anyone help me fix this issue?
Step 2 (using amix suggested by kesh)
ffmpeg -safe 0 -f concat -segment_time_metadata 1 -i list.txt -stream_loop -1 -i track1.mp3 -filter_complex "[0:a]volume=1.5[a0];[1:a]volume=0.03[a1];[a0][a1]amix=inputs=2:duration=first[a]" -map 0:v -map "[a]" -c:v copy -ac 2 out.mp4
And, following is the full log of step 2,
ffmpeg version 5.0.1-full_build-www.gyan.dev Copyright (c) 2000-2022 the FFmpeg developers
built with gcc 11.2.0 (Rev7, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libmfx --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil 57. 17.100 / 57. 17.100
libavcodec 59. 18.100 / 59. 18.100
libavformat 59. 16.100 / 59. 16.100
libavdevice 59. 4.100 / 59. 4.100
libavfilter 8. 24.100 / 8. 24.100
libswscale 6. 4.100 / 6. 4.100
libswresample 4. 3.100 / 4. 3.100
libpostproc 56. 3.100 / 56. 3.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001ff7163ed40] Auto-inserting h264_mp4toannexb bitstream filter
Input #0, concat, from 'list.txt':
Duration: N/A, start: -0.023220, bitrate: 63 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 854x480, 61 kb/s, 29.97 fps, 29.97 tbr, 30k tbn
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 1 kb/s
Metadata:
handler_name : SoundHandler
vendor_id : [0][0][0][0]
Input #1, mp3, from 'track1.mp3':
Metadata:
title : Safety Net
artist : Riot
album : YouTube Audio Library
genre : Country & Folk
encoder : Google
Duration: 00:01:30.88, start: 0.025057, bitrate: 128 kb/s
Stream #1:0: Audio: mp3, 44100 Hz, stereo, fltp, 128 kb/s
Metadata:
encoder : Lavf
Stream mapping:
Stream #0:1 (aac) -> volume:default
Stream #1:0 (mp3float) -> volume:default
Stream #0:0 -> #0:0 (copy)
amix:default -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
Output #0, mp4, to 'out.mp4':
Metadata:
encoder : Lavf59.16.100
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 854x480, q=2-31, 61 kb/s, 29.97 fps, 29.97 tbr, 30k tbn
Metadata:
handler_name : VideoHandler
vendor_id : [0][0][0][0]
Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 128 kb/s
Metadata:
encoder : Lavc59.18.100 aac
frame= 3 fps=0.0 q=-1.0 size= 0kB time=00:00:00.02 bitrate= 16.5kbits/s speed=44.7x
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001ff719acac0] Auto-inserting h264_mp4toannexb bitstream filter
frame= 478 fps=0.0 q=-1.0 size= 256kB time=00:00:15.89 bitrate= 132.0kbits/s speed=31.7x
frame= 1084 fps=1083 q=-1.0 size= 1024kB time=00:00:36.11 bitrate= 232.3kbits/s speed=36.1x
frame= 1668 fps=1111 q=-1.0 size= 1536kB time=00:00:55.59 bitrate= 226.3kbits/s speed= 37x
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001ff719acac0] Auto-inserting h264_mp4toannexb bitstream filter
frame= 2282 fps=1140 q=-1.0 size= 2560kB time=00:01:16.08 bitrate= 275.6kbits/s speed= 38x
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001ff71ce81c0] Auto-inserting h264_mp4toannexb bitstream filter
frame= 2848 fps=1138 q=-1.0 size= 3328kB time=00:01:34.97 bitrate= 287.1kbits/s speed=37.9x
frame= 3443 fps=1146 q=-1.0 size= 4352kB time=00:01:54.82 bitrate= 310.5kbits/s speed=38.2x
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001ff718f2c40] Auto-inserting h264_mp4toannexb bitstream filter
[mov,mp4,m4a,3gp,3g2,mj2 @ 000001ff718f26c0] Auto-inserting h264_mp4toannexb bitstream filter
[aac @ 000001ff716ed2c0] Queue input is backward in time
[mp4 @ 000001ff71661600] Non-monotonous DTS in output stream 0:1; previous: 6224608, current: 6221946; changing to 6224609. This may result in incorrect timestamps in the output file.
[mp4 @ 000001ff71661600] Non-monotonous DTS in output stream 0:1; previous: 6224609, current: 6222970; changing to 6224610. This may result in incorrect timestamps in the output file.
[mp4 @ 000001ff71661600] Non-monotonous DTS in output stream 0:1; previous: 6224610, current: 6223994; changing to 6224611. This may result in incorrect timestamps in the output file.
frame= 4336 fps=1238 q=-1.0 size= 5376kB time=00:02:24.65 bitrate= 304.4kbits/s speed=41.3x
frame= 4406 fps=1230 q=-1.0 Lsize= 5783kB time=00:02:27.07 bitrate= 322.1kbits/s speed=41.1x
video:3486kB audio:2149kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 2.641165%
[aac @ 000001ff716ed2c0] Qavg: 853.082