How to avoid a Nest.Js / Node.Js process taking up 100% of the CPU?

user_712347 0
Viewed 1125

I have an app running on Nest.Js / Node.Js which does text processing and because of that it has an .map (or .forEach) iteration that takes a lot of resources (tokenizing a sentence, then removing the stopwords, etc — for each sentence of which there may be tens of thousands).

For reproducibility, I provide the code I use below, without the text processing details — just a long heavy loop to emulate my problem:

  const before = Date.now() 
  const statements = [...Array(100)]
  let l = 0
  let p = 0
  const processed = 
     statements.map((value, index) => {
      
         // a set of consecutive functions that take a long time to execute
         for (let i = 0; i < 100000; i++) {
              l = l + i * Math.random()
         }
         // console.log(index + ' ' + l + ' ' + (Date.now() - before))
             p = index
       
    
        return l
     })
  
  console.log(processed)

  const after = Date.now()
  console.log(p + ' results in ' + (after - before) + ' ms')

For very big files the process takes up to 100% of the CPU, so the app becomes unresponsive.

I was wondering if you know of any solution to prevent the app from taking all the CPU made available to it. Like is there a way to set any single process to take 80% and always leave 20% for the other users?

PS I run it from Amazon's ECS so of course I can create clusters and also use PM2 for clusterization, but I wonder how I can avoid the 100% load from a single process within a Javascript / Node.Js app itself. I mention the Nest.Js framework because maybe there's a solution there as well.

nestjs node.js foreach javascript performance
2 Answers

In terms of limiting a single thread from using 100% CPU, there are architectural ways of doing so at a server level, however I don't think that's really the outcome you would want. A CPU using 100% isn't an issue (CPUs will often spike to 100% CPU for very short periods of time to process things as quickly as possible), it's more of it using 100% CPU for an extended period of time and preventing other applications from getting CPU cycles.

From what I am seeing in the example code, it might be a better solution to use Queues within NestJS. Documentation can be seen here using Bull. This way you can utilize the rate limits of jobs being processed and tweak it there, and other applications will not be waiting for the completion of the entire process.

For instance if you have 100,000 files to process, you may want to create a job that will process 1,000 of them at a time and create 100 jobs to be thrown into the queue. This is a fairly typical process for processes that require a large amount of compute time.

I know this isn't the exactly the answer I am sure you were looking for, but hopefully it will help and provide a solution that is not specific to your architecture.

user_46231640

JS is a syncronous language. Now when using this on your node application, you can end up blocking your main thread if not done correctly.

https://nodejs.org/en/docs/guides/dont-block-the-event-loop/

Node.js has two types of threads: one Event Loop and k Workers. The Event Loop is responsible for JavaScript callbacks and non-blocking I/O, and a Worker executes tasks corresponding to C++ code that completes an asynchronous request, including blocking I/O and CPU-intensive work. Both types of threads work on no more than one activity at a time. If any callback or task takes a long time, the thread running it becomes blocked. If your application makes blocking callbacks or tasks, this can lead to degraded throughput (clients/second) at best, and complete denial of service at worst.

To write a high-throughput, more DoS-proof web server, you must ensure that on benign and on malicious input, neither your Event Loop nor your Workers will block.

One request is to offload the operation to another area.

  • for example can you do part of the operation on Database
  • like if you are working on mongodb, can mongodb manage the replaceAll in aggregation, or do some other manipulation
  • this way you dont have to do this at your node application level
user_89633570
Related