QuerySelector's returning an empty selection in console while on browser side it contains elements

user_17395445 0
Viewed 22

In order to learn web scraping with puppeteer , i have started a little project , which aims to extract the planning of Power Outages from the National Power Supplier's website. In order to do that i have to manually change the region then retrieve the Outage's program list. The QuerySelector request i use browser side looks totally fine as it contains without fault all the outages displayed . But when i use it on the server end i receive an empty list. Here is my code and the url of the website can be found in it . Thanks in advance !

const puppeteer = require('puppeteer');

(async() => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto('https://alert.eneo.cm/', { waitUntil: 'networkidle0' });

await page.evaluate(() => {
    var region = "Littoral";
    var j = $('#regions option:contains(' + region + ')');
    $('#regions').val(j.val()).change();
});

const outages = await page.evaluate(() => {

    const elements = document.querySelectorAll("#contentdata .outage");
    return elements;
});
console.log(outages);

})();

puppeteer node.js
1 Answers

I see there is list of power outages on the page you want scrape. Here is how you can get the power outage data for the first div

(async()=>{
    let browser = await puppeteer.launch();
    let page = await browser.newPage();
    await page.goto('https://alert.eneo.cm/', { waitUntil: 'networkidle0' });
    await page.select('select[name="regions"]', '5')
    const outageData = await page.evaluate( async () => {
      let quartier = document.querySelector('div[class="quartier"]').innerText;
      let ville = document.querySelector('div[class="ville"]').innerText;
      let observations = document.querySelector('div[class="observations"]').innerText;
      let dateAndTime= document.querySelector('div[class="prog_date"]').innerText;
      return {quartier, ville, observations, dateAndTime}
    });
    await browser.close();
    console.log(outageData);
})();
user_148889230
Related