Buffer as input and output for fluent-ffmpeg

Question

The below looks like a lot but a it's primarily just output.

I'm trying to take in a buffer using multer (being send the request containing the video (.mov format) through Alamofire from an iPhone) as the input before a fluent-ffmpeg conversion, then I want it to output as a buffer, and then send the result to S3. I think I'm close, but I don't think fluent-ffmpeg can have a buffer passed in. This is deployed on heroku using this buildpack: https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.... How do I pass it in correctly?

const multer = require('multer')
const upload = multer({ limits: { fieldSize: 100_000_000 } })
app.post('/create', upload.single('video'), async function (request, response, next) {
  let data = request.body
  console.log(data) // prints [Object: null prototype] { param1: '' }
  let bufferStream = new stream.PassThrough();
  console.log(request.file.buffer) // prints ''

  new ffmpeg({
    source: stream.Readable.from(request.file.buffer, { objectMode: false })
  })
  // .format('mp4')
  .on('error', function (err) {
    console.log(Error: ${err})
  })
  .on('progress', function (progress) {
    console.log("progress")
  })
  .on('end', function () {
    console.log('Formatting finished!');
    console.log("after");
  })
  .writeToStream(bufferStream);

  // Read the passthrough stream
  const buffers = [];
  bufferStream.on('data', function (buf) {
    buffers.push(buf);
  });
  bufferStream.on('end', function () {
    const outputBuffer = Buffer.concat(buffers);
  // use outputBuffer
  });
  console.log("Added.")
  response.send("Success")
});

The output is what you can see below (without .format('mp4')):

2022-09-03T13:12:24.194384+00:00 heroku[router]: at=info method=POST path="/createBusiness" host=sparrow-testing.herokuapp.com request_id=2774a4ec-e21e-4c2f-8086-460a4cba7d1d fwd="74.71.236.5" dyno=web.1 connect=0ms service=13157ms status=200 bytes=762 protocol=https
2022-09-03T13:12:24.186257+00:00 app[web.1]: [Object: null prototype] { title: '' }
2022-09-03T13:12:24.187296+00:00 app[web.1]: 
2022-09-03T13:12:24.189891+00:00 app[web.1]: Added.
2022-09-03T13:12:24.891564+00:00 app[web.1]: Error: Error: ffmpeg exited with code 1: pipe:1: Invalid argument
2022-09-03T13:12:24.891570+00:00 app[web.1]:

This output is what you see with .format('mp4'):

2022-09-03T13:17:07.380415+00:00 app[web.1]: [Object: null prototype] { title: '' }
2022-09-03T13:17:07.381335+00:00 app[web.1]: 
2022-09-03T13:17:07.384047+00:00 app[web.1]: Added.
2022-09-03T13:17:07.388457+00:00 heroku[router]: at=info method=POST path="/createBusiness" host=sparrow-testing.herokuapp.com request_id=84e69ead-09b1-4668-8fc8-b9fc9d5f229d fwd="74.71.236.5" dyno=web.1 connect=0ms service=13079ms status=200 bytes=762 protocol=https
2022-09-03T13:17:08.339746+00:00 app[web.1]: Error: Error: ffmpeg exited with code 1: Conversion failed!
2022-09-03T13:17:08.339783+00:00 app[web.1]:

My uploadFile function works correctly because I use it elsewhere--normally, I just pass in the request.file.buffer, but here it needs to be a buffer after the ffmpeg conversion

EDIT:

At Heiko's suggestion, I tried changing the multer initialization to

multer({ limits: { fieldSize: 100_000_000 }, dest: "uploads/" })

and the source I was passing in to ffmpeg to

new ffmpeg({
  source: request.file.path // request.file.path seems to be a path of a Multer-generated file, I'd assume the one I'm sending to the server
})
.format('mp4')

but it still errored out to

Error: ffmpeg exited with code 1: Conversion failed!

when the request.file was:

{
  fieldname: 'video',
  originalname: 'video',
  encoding: '7bit',
  mimetype: 'clientMime',
  destination: 'uploads/',
  filename: '08d5d3bbdcf1ac29fb97800136a306e9',
  path: 'uploads/08d5d3bbdcf1ac29fb97800136a306e9',
  size: 1567480
}

user_8807231 · Answer

I give a try with your code and i get same result. After searching a while, i found an explanation.

Well, problem is around MP4 and streams.

I am not a specialist of video, but it seems that when manipulating MP4 file, ffmpeg need to seek into file ans seeking is incompatible with streams by nature.

So you need to output a filesystem file from ffmpeg (make this file temporary by removing it after using it) and then use this file according to your needs.

For having more debug information about ffmpeg error you need to modify your error event listener code like this:

// .format('mp4')
  .on('error', function (err, stdout, stderr) {
    console.log(Error: ${err})
    console.log('Stdout: %o', stdout);
    console.log('Stderr: %o', stderr);
  })
  .on('progress', function (progress) {

And you will see while executing the stderr logs of the ffmpeg process.

You can have confirmation of this here:

https://stackoverflow.com/a/39679975/8807231

https://superuser.com/a/760287

user_11760058 · Answer

MP4, MOV and 3GP all contain some important metadata at the end of the file. This stems from ages predating streaming and makes these formats by default unfit for it.

The solution is to store that so called moov atom at the beginning of the file. This means you must preprocess iPhone videos in order to stream them.

With ffmpeg e.g. via FFmpegKit the option is -movflags faststart.
With AVAssetWriter/AVAssetExportSession it's the flag shouldOptimizeForNetworkUse (via).
These need to write to a file on device though, maybe you can work around that with a named pipe read by Alamofire.

VLC and iOS separately transfer the moov atom to make sense of the rest. If you could apply this strategy to ffmpeg you'd only need to find the moov atom on the phone. But there were no examples to find of ffmpeg setup this way.

Your server code seems fine, probably only needs the -movflags faststart as output option there, too. For testing upload an MKV or WEBM file which ffmpeg can read streamed out of the box.

Buffer as input and output for fluent-ffmpeg

2 Answers