puppeteer: Getting HTML from NodeList?

user_1055279 0
Viewed 143

I'm getting a list of 30 items from the code:

const boxes = await page.evaluate(() => {
    return document.querySelectorAll("DIV.a-row.dealContainer.dealTile")
})

console.log(boxes);

The result

{ '0': {},
  '1': {},
  '2': {},
  ....
  '28': {},
  '29': {} }

I have the need to see the html of the elements.

But every property I tried of boxes is simply undefined. I tried length, innerHTML, 'innerText` and some other.

I am sure of box really containing something because puppeteer's screenshot shows the content before I start to 'browse' the content of the page

What am I doing wrong?

puppeteer node.js web-scraping javascript
1 Answers

There are multiple ways to do this:

  • Use page.$$eval to execute the selector and return the result in one step.
  • Use page.evaluate to get the attributes after querying the elements.

Code sample for page.$$eval

const htmls = await page.$$eval('selector', el => el.innerHTML);

Code sample for page.evaluate

const singleBox = boxes[0];
const html = await page.evaluate(el => el.innerHTML, singleBox);
user_56275990
Related