I'm building a Node.js + Express API REST app. So long the CPU and RAM usage is on normal levels, but one of the latest endpoints designed is taking up too much RAM and CPU:
I'm talking about an API whose goal is to generate .pdf files in real time using template data (we're using the library pdf-puppeteer). But when this API is tasked to generate hundreds of .pdf's the Node.js application goes unresponsive and we cannot call the other API's as they give either a timeout error or they take too long to give a response, even the simpler ones.
I'm using pm2 for load balancing, and we've tried to delegate the pdf creation process to worker processes so the event loop doesn't get blocked. And it was succesful to some extent, but still the CPU and RAM consumption is very high and the API's start to get unresponsive neverthless.
So how can this high CPU and RAM usage be prevented on heavy processes, so the application doesn't get unresponsive? Maybe using a throttling approach?