When should I use worker-threads?

user_6098257 0
Viewed 750

I am currently working on a backend which provides rest endpoints for my frontend with nestjs. In some endpoints I receive e.g. an array of elements which I need to process.

Concrete Example: I receive an array of 50 elements. For each element I need to make a SQL request. Therefore I need to loop over the array and do stuff in SQL.

I always ask myself: At what amount of elements should I use for example worker threads to not block the event loop?

Maybe I misunderstood the blocking of the event loop and someone can enlight me.

nestjs node.js
2 Answers

I don't think that you'll need worker-threads in this scenario. As long as the sql-queries are executed asynchronsouly, i.e. the sql-query calls do not block, you will be fine. You can use Promise.all to speed up the processing of the loop, as the queries will be executed in parallel, e.g.

const dbQueryPromises = [];

for(const entry of data) {
   dbQueryPromises.push(dbConnection.query(buildQuery(entry)));
}

await Promise.all(dbQueryPromises);

If, however, your code performs computation-heavy operations inside the loop, then you should consider worker-threads as the long running operations on your call stack will block the eventloop.

user_37616280

Only use them if you need to do CPU-intensive tasks with large amounts of data. They allow you to avoid the serialization step of the data. 50 Is not enough I believe

user_112309080
Related