Puppeteer page.goto not working for query parameters

user_7618760 0
Viewed 2375

Using Puppeteer, I'm trying to navigate to a new page for different sets of query parameters. Puppeteer updates the URL's query parameters, but doesn't navigate to a new page. This seems to be the expected behavior, since it takes pressing enter twice to go to a new page in my browser when testing manually.

Here's the line of code:

await page.goto(`https://www.google.com/get/sunroof/building/${lat}/${lng}/#?f=${type}&b=${amount}`);

Changing lat and lng goes to new pages... type and amount does not.

Bonus: An alternative to navigating to a new page might be be to call the function(s) on the webpage that handle the query parameter data. They're bound to buttons and a dropdown on the page... the parameters appear in the URL when the buttons and dropdown are clicked.

The app is Angular version 1.6.4. I'm not sure whether this is possible... but would be awesome if it was!

puppeteer node.js web-scraping
1 Answers

It is indeed the expected behaviour with AngularJs apps.

If you are interested in workarounds: then if you visit 'about:blank' between two pages with different type and amount parameters you will be able to reload the content. You just need to use page.goto between each query change.

const puppeteer = require('puppeteer')

async function getSunroofPages() {
  const browser = await puppeteer.launch({ headless: false })
  const page = await browser.newPage()
  let type
  let amount

  // first visit
  try {
    type = 'lease'
    amount = 5000
    await page.goto(`https://www.google.com/get/sunroof/building/36.7396315/-119.7842608/#?b=${amount}&f=${type}`)
  } catch (e) {
    console.error(e)
  }

  // the trick
  await page.goto('about:blank')

  // second visit
  try {
    type = 'buy'
    amount = 80000
    await page.goto(`https://www.google.com/get/sunroof/building/36.7396315/-119.7842608/#?b=${amount}&f=${type}`)
    await page.waitFor(10000) // to give you time to check it is a different content
  } catch (e) {
    console.error(e)
  }
  await browser.close()
}
getSunroofPages()
user_124125950
Related