Cannot get the text out of node by using XPath

Question

I am trying to get the last movie/series I rated on Letterboxhd by using XPath, then print it. To get the first movie, I found this in HTML: "Magnolia (1999)". To get the rating of the first movie: "★★★" I know that every time I run this code the rating will be 3 stars, so I only wrote half of it.

Here is what I did:

let data = await page.evaluate(() => {

        let titles = document.evaluate("//span[contains(@class, 'frame-title')]", document, null, XPathResult.ANY_TYPE, null);
        let title = titles.iterateNext();

        let ratings = document.evaluate("//span[contains(@class, ' -tiny')]", document, null, XPathResult.ANY_TYPE, null);
        let rating = ratings.iterateNext(); 


        return{
            title,
            rating
        }
    });

When I run this code, I see "data" is undefined. What am I doing wrong? What should I do instead?

Here is my entire code:

const puppeteer = require('puppeteer');

(async () => {

    let movieUrl = 'https://letterboxd.com/sdeer/films/';
    
    let browser = await puppeteer.launch({ headless: true });
    let page = await browser.newPage();

    await page.goto(movieUrl, { waitUntil: 'networkidle2'});

    let data = await page.evaluate(() => {

        let titles = document.evaluate("//span[contains(@class, 'frame-title')]", document, null, XPathResult.ANY_TYPE, null);
        let title = titles.iterateNext();

        let ratings = document.evaluate("//span[contains(@class, ' -tiny')]", document, null, XPathResult.ANY_TYPE, null);
        let rating = ratings.iterateNext(); 


        return{
            title,
            rating
        }
    });

    debugger

    console.log(data.title.textContent);
    console.log(data.rating.textContent);

    

    await browser.close();

})();

user_252228 · Answer

In terms of first versus last title I think using let lastTitle = document.evaluate('descendant::span[contains(@class, "frame-title")][last()]', document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue is a better way to read out the last span in document order that has the class as frame-title.

Nevertheless, it looks as if the web site uses paging so this would give you the last title on the first page.

As for passing DOM nodes around, I think for that you need page.evaluateHandle instead of page.evaluate.

I think for the the evaluate method you can pass JSON on:

let data = await page.evaluate(() => {

        let title = document.evaluate("//span[contains(@class, 'frame-title')]", document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue.textContent;
        

        let rating = document.evaluate("//span[contains(@class, ' -tiny')]", document, null, XPathResult.FIRST_ORDERED_NODE_TYPE, null).singleNodeValue.textContent;


        return {
            title,
            rating
        }
    });

Cannot get the text out of node by using XPath

1 Answers