nodejs/express and binary data in POST

user_389812 0
Viewed 23897

I'm trying to send binary data to an express app. It works fine, as long as my values are smaller than 0x80. If a single value is 0x80 or greater, it messes up the whole buffer.

Express Handler:

binary = require('binary');

exports.api = function(req, res){
    var body = req.body.name;
    var buf = new Buffer(body,'binary');

    console.log('body',req.body);
    console.log('req body len', body.length);
    console.log('buf len', buf.length);

    var g = binary.parse(buf)
        .word16bu('a') // unsigned 16-bit big-endian value
        .word16bu('b').vars

    console.log('g.a', g.a);
    console.log('g.b', g.b);

  res.send("respond with a resource");
};

Python client (Content-Type: application/x-www-form-urlencoded):

import requests
from struct import pack
# send two unsigned shorts (16-bits each).
requests.post('http://localhost:3000/api', data={'name':pack('!HH',1,2)})

Express output when data = 1,2. This is what I'd expect.

body { name: '\u0000\u0001\u0000\u0002' }
req body len 4
buf len 4
g.a 1
g.b 2
POST /api 200 1ms - 23b

Express output when data = 1,0xFF. It's interesting to note that 9520 is actually 0x25 0x30 in hex, which corresponds to "%0" in ASCII. Yes, it appears to be parsing the '%00%01...' string. I wish I knew how to prevent this!!!

body { name: '%00%01%00%FF' }
req body len 12
buf len 12
g.a 9520
g.b 12325
POST /api 200 2ms - 23b
express node.js binary
3 Answers

After struggling with this for way too long, I came up with this solution:

var express = require('express');
var bodyParser = require('body-parser');
var fs = require('fs');
var path = require('path');

app.use(bodyParser.raw({type: 'application/octet-stream', limit : '2mb'}))

app.post('/nist-ws/rest/v1/nist/', function(req, res) {
    var filePath = path.join(__dirname, 'nist_received', `/${Date.now()}.nist`)
    fs.open(filePath, 'w', function(err, fd) {  
        fs.write(fd, req.body, 0, req.body.length, null, function(err) {
            if (err) throw 'error writing file: ' + err;
            fs.close(fd, function() {
                console.log('wrote the file successfully');
                res.status(200).end();
            });
        });
    });
});

bodyParser.raw fills req.body with a Buffer and the rest of the code is from: https://stackabuse.com/writing-to-files-in-node-js/

user_5558830

A modern Typescript safe way would probably look like this:

export async function readBodyAsBuffer(req: any): Promise<Buffer> {
  return new Promise((resolve, reject) => {
    let buffer = Buffer.alloc(0)
    req.setEncoding(null)
    req.on(
      "data",
      (chunk: string) => (buffer = Buffer.concat([buffer, Buffer.from(chunk)]))
    )
    req.on("end", () => resolve(buffer))
    req.on("error", reject)
  })
}

You can now simply get the body as Buffer like this:

let buffer = await readBodyAsBuffer(req)
user_1409270
Related