Puppeteer behaving differently headless

user_14668053 0
Viewed 399

I have the following script

const puppeteer = require('puppeteer');

async function startBrowser() {
    const browser = await puppeteer.launch({headless: false});
    const page = await browser.newPage();
    return {browser, page};
}

async function closeWindows(page) {
    await page.waitForSelector(selector1);
    await page.click(selector1);
    await page.waitForSelector(selector1);
    await page.click(selector2);
}
async function doStuff(URL) {
    const {browser, page} = await startBrowser();
    await page.goto(URL);
    await closeWindows(page);
}

doStuff(URL)

Where selector1, selector2 and URL are defined.

This works fine, but as soon as I try to run it in headless mode, I get the following error message: TypeError: Cannot read property 'waitForSelector' of undefined. I already tried using

await page.waitForNavigation({ waitUntil: 'networkidle0' })

but it just keeps me waiting until I get a TimeoutError. If this helps: the URL is a youtube video.

puppeteer node.js javascript
2 Answers

I solved this by adding the following line:

await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36');

Apparently, the website was being opened in mobile mode.

user_146680530

Uhm that seems strange, even if sometimes I myself noticed some discrepancies between the two modes. For example with a customer's website using a React app puppeteer was crashing or something like that.
Instead of using waitForNavigation, I'd use the options that goto offers you (see docs here).
So it should be something like:

 await page.goto('https://news.ycombinator.com', { 
     waitUntil: ['load', 'networkidle2'],
     timeout: 10000,
 });

This method also returns a HTTPResponse object where you can inspect the status code and the body as text.
Take also a look at the debugging tips here, for example intercepting console messages and outputting puppeteer's messages from the DEBUG module.

user_13632650
Related