Getting ERR_FS_FILE_TOO_LARGE while using unirest file send with put

Viewed 388

I am using unirest to upload a file like so

 unirest.put(fullUri)
    .auth({
      user: self.userName,
      pass: self.password
    })
    .header('X-Checksum-Sha1', sha1Hash)
    .header('X-Checksum-Md5', md5Hash)
    .send(fs.readFileSync(filePath))
    .end(function (response) {

This works fine for smaller files but for large files I get ERR_FS_FILE_TOO_LARGE error. I have already tried max_old_space_size without success. Looks like I can fix this by streaming the file but I can't find an api to do that in unirest js library.

3 Answers

Streaming files with unirest is available via the .attach method:

 unirest.put(fullUri)
    .auth({
      user: self.userName,
      pass: self.password
    })
    .header('X-Checksum-Sha1', sha1Hash)
    .header('X-Checksum-Md5', md5Hash)
//    .send(fs.readFileSync(filePath))
    .attach('filename', fs.createReadStream(filePath))
    .end(function (response) {

I can't find an API to do that in unirest js library.

That's because there is none: https://github.com/Kong/unirest-nodejs/issues/49:

  • You can use the underlying request library to do streaming if you want, I am open to a pull request either on this branch or the 1.0 version to add streaming.

Issue is still open.


But from this issue and from the source code you can find out that end() returns request (see https://github.com/request/request)

Unirest.request = require('request')

...

  end: function (callback) {
    ...
    Request = Unirest.request($this.options, handleRequestResponse)
    Request.on('response', handleGZIPResponse)
    ...
    return Request
  }

and from request's source code you can find out that no actual request is sent yet (it's defered). So you can hack into it. And use it's API instead:

const request = unirest.put(constants.server2)
  .auth({
    user: self.userName,
    pass: self.password
  })
  .header('X-Checksum-Sha1', sha1Hash)
  .header('X-Checksum-Md5', md5Hash)
  // .send(fs.readFileSync(filePath))
  .end(...)

fs.createReadStream(filePath).pipe(request) // just pipe it!

As a side note: unirest is based on request, request is deprecated now. So... maybe you need to steer away from unirest.

Related