Promises execute when defined instead of Promises.all?

user_4634109 0
Viewed 322

I am trying to create an array of promises so I can batch promises instead of executing them all together to avoid shutting down my server. I want to execute up to 20 promises at a time. I have written the following code:

let promises = [];
let create;
let update;
let i=0;
let users = users.json;

if (users && usersjson.users.length) {
  for (const user of users) {
    if (userexists) {
      update = new promise(async (resolve,reject) => {
        //Function to update user
      });
      promises.push(update);
      i++;
    } else {
      create = new promise (async (resolve,reject) => {
        //Function to create a new user
      });
      promises.push(update);
      i++;
    }
  if (promises.length >= 20 || i >= usersjson.users.length) {
    await Promise.all(promises)
    .then((response)=>{
    console.log(response);
    promises=[];
  }).catch((err)=> {
    console.log(err);
  })
}
}
}

However, I found that the promises are executed when I define them, instead of being pushed into the array and executed when I call Promise.all, and I can't figure out why.

I would also prefer to have the Promise.all function continue to run even if a single promise was rejected, if that's possible with the way my code is built. I have a catch inside each promise in case the code fails, is that the correct way to do that?

promise.all promise async-await node.js arrays
2 Answers

As mentioned in comments, promise objects actually get in "pending" state right after you executed an asynchronous operation and are "fulfilled" as soon as operation finishes.

So in your code you actually create and run async operations for all users, not only 20 of them.

There are two solutions for you. The main one is to create functions which return a promises and run only 20 at one time.

const jobs = users.map(user => () => user.update(...)); // array of functions
while (jobs.length > 0) {
   await Promise.all(jobs.splice(0,20).map(job => job())); // take 20 and process
}

An alternative solution is to use a library like bluebird which has a lot of useful methods for working with promises. One you'd probably like is map() which support concurrency limit.

Your second question was regarding errors in Promise.all what results in a fail of the whole series. In order to prevent that you may add .catch() to each job and, for example, return a null there or any other value which then will help you to determine that some operations failed. Of course, this approach will also prevent Promise.all from interrupting.

const jobs = users.map(user => () => user.update().catch(e => null));
user_50757550

A function in Javascript executes directly. Difference with a Promise is that you can wait for it in a Promise-style way. When the function is called, so like you're doing in the for loop, it executes it right away. Later on, with the Promise.all, you're just saying: "Wait for the promises I've defined in my array to complete" (and error if one of them fails).

About your second question, there's a way to wait for all promises to complete, and not throwing an error (and thus stopping). That's with Promise.allSettled. As the name suggests, it waits for all promises to settle.

However, keep in mind that the response will be different than the Promise.all function. As stated in the docs, it'll return an array per promise if the function is 'fulfilled' or 'rejected'.

user_77202100
Related