Stream Promise results to client as they resolve in Promise.all

user_5582330 0
Viewed 594

In Node, I am using bluebird's Promise.all to execute Promises asynchronously. I don't want to wait for all Promises to resolve before I send results back to the client; rather, I'd like to stream the result of each Promise as soon as it resolves. Is this possible? My research indicates that it is not, but I thought it worth reaching out to the community.

Here is the code (with no attempt to implement streaming):

async bulkExecution(req, res) {
    try {
        const { assets } = req.body;

        let thePromises = _.map(assets, (asset) => {
            return onePromise(asset);
        });

        // I want to stream each Promise result to client as it resolves
        let results = await Promise.all(thePromises);

        return res.status(200).send(results);
    } catch (err) {
        return res.status(500).send(err);
    }
}
promise node.js stream
2 Answers

I think you'd have to iterate over all the promises and do a .then() on each promise and write a partial response then close the response after all the promises complete. Something like this should work. The order of the responses may be different than the order of the promise array. For instance, my example below will return the second promise first.

let res = {
  write: (data) => {console.log(data)},
  end: () => {}
};

let promise1 = new Promise(function(resolve, reject) {
  setTimeout(function() {
    resolve('foo');
  }, 500);
});

let promise2 = new Promise(function(resolve, reject) {
  setTimeout(function() {
    resolve('foo');
  }, 300);
});

let promises = [promise1, promise2];

let count = 0;
promises.forEach((promise) => {
  promise.then((data) => {
    res.write(data);
    count++;
    if (count >= promises.length){
        res.end();
    }
  })
})

Modified to write in JSON format.

let res = {
  write: (data) => {
    console.log(data)
  },
  end: () => {}
};

let promise1 = new Promise(function(resolve, reject) {
  setTimeout(function() {
    resolve({foo:"bar"});
  }, 500);
});

let promise2 = new Promise(function(resolve, reject) {
  setTimeout(function() {
    resolve({bar:"foo"});
  }, 300);
});

let promises = [promise1, promise2];

let count = 0;

res.write("[");
promises.forEach((promise) => {
  promise.then((data) => {
    res.write(JSON.stringify(data));
    count++;
    if (count >= promises.length) {
      res.write("]");
      res.end();
    } else {
      res.write(",");
    }
  })
})

Streaming out JSON like this isn't common, but could be used to reduce the amount of memory the server uses. That's because you don't need to create the full response in memory before sending the response. The client will still need enough memory to read the entire object into memory.

user_1575410

Try doing .then( data => data.json) on the promise like so

    let thePromises = _.map(assets, (asset) => {
        return onePromise(asset).then(data => data.json);
    });
user_54396690
Related