Puppeteer : script seems to go iddle

Viewed 30

I am currently working on a script that takes screenshots of every individual div inside of a page. The script works but is getting slower after each iteration. For example, the 11th iteration takes several minutes, while the first takes only but a few seconds. Also, the script never actually stops with an error, I believe it will run forever if I don't interrupt it.

Did any of you guys ever went through the same issue ? Is there a way to solve this problem ? I am new to promises and Puppeteer...

Thanks in advance !

Below is the code of my function and the execution output.

Here is the code of the function :

async function takeScreenshot(bookPage){
    await bookPage.evaluate(() => {
        document.body.style.position = "relative";
        document.querySelector("#page-container").style.position = "relative";
     });

    for(var i = 0; i < allPagesDimensions.length; i++){
        console.log("Itération n°" + i);

        await bookPage.evaluate((i) => {
            const page = document.querySelector("#pf" + (i + 1));
            window.scrollTo(0, page.offsetTop);
        }, i);

        const pageDimensions = await bookPage.evaluate((i) => {
            const page = document.querySelector("#pf" + (i + 1));
            var returnValue = { x: page.offsetLeft, y: page.offsetTop, width: page.clientWidth, height: page.clientHeight };
            return returnValue;
        }, i);

        console.log("Lancement de la capture");
        await bookPage.screenshot({
                path: "page-" + i + ".png",
                clip: { "x": pageDimensions.x, "y": pageDimensions.y, "width": pageDimensions.width, "height": pageDimensions.height }
        });
        console.log("Capture réussie");
    }
}

And the output is the following :

Itération n°0
Lancement de la capture
Capture réussie
Itération n°1
Lancement de la capture
Capture réussie
Itération n°2
Lancement de la capture
Capture réussie
Itération n°3
Lancement de la capture
Capture réussie
Itération n°4
Lancement de la capture
Capture réussie
Itération n°5
Lancement de la capture
Capture réussie
Itération n°6
Lancement de la capture
Capture réussie
Itération n°7
Lancement de la capture
Capture réussie
Itération n°8
Lancement de la capture
Capture réussie
Itération n°9
Lancement de la capture
Capture réussie
Itération n°10
Lancement de la capture
Capture réussie
Itération n°11
1 Answers

I finally managed to achieve this bot.

The trick was to wrap the calls to asynchronous functions inside a try-catch and go back at the top of the loop when the methods throw an exception. This way, the bot will try again and again until puppeteer eventually manages to do what you ask him to.

Worked with huge pages.

Related