Puppeteer: New Browser or new Page? Which is more memory expensive?

Viewed 174

I'm running a heroku free dyno with 512 MB (I plan to upgrade later) of RAM, on which I'm running a express server to host my api with returns the 'src' attribute of images on the webpage that I want to scrape. Currently my express route looks like this

async function run(){
            let browser;
            try{
                browser = await puppeteer.launch({
                    args: [
                        '--no-sandbox',
                        '--disable-setuid-sandbox',
                    ],
                });
                const page = await browser.newPage();
                await page.setViewport({ width: 1200, height: 1200 });
                await page.goto(url,{waitUntil:'load',timeout: 0});
                const imgs = await page.$$eval('.img-wrapper img[src]', imgs => imgs.map(img => img.getAttribute('src')));
                res.send({imageList:imgs})
            }catch(e){
                res.send({errormsg:"Something went wrong with the scrapper"});
                console.log(e)
            }finally{
                await browser.close(); 
            }
        }

so each time I send a POST request to the express route run() executes.

But the problem is if multiple post request is send (say more than 100) I assume 100 browser instances are opened up and closed (I assume as I used async but I do know JS is synchronous, so I'm not sure), which means I run out of memory and heroku will stop my dyno automatically.

So I was wondering if it is better to declare the browser instance as a global variable and then open and close tab/pages instead, and have the browser always running.

Something like

const browser = await puppeteer.launch({
                    args: [
                        '--no-sandbox',
                        '--disable-setuid-sandbox',
                    ],
                });

app.post(....

const page = await browser.newPage();
await page.goto(url,{waitUntil:'load',timeout: 0}); // openpage
//do stuff

finally{
page.close()

})

0 Answers
Related