Puppeteer Connect through IP

Viewed 2189

I have a problem with connecting through IP to certain website. I install Google Chrome using Docker and then connect to the current instance using puppeteer.connect():

(async () => {
  try {
    const IP = '104.20.0.109'

    const browser = await puppeteer.connect({
      browserURL: 'http://localhost:9222' //connect to Google Chrome
    })

    const page = await browser.newPage()

    await page.setExtraHTTPHeaders({ host: 'www.skillshare.com'}) //configure host to connect via IP
    await page.setUserAgent('MyUserAgent')

    await page.goto(`http://${IP}`, { waitUntil: 'networkidle0' })

    console.log('Yeah, it works!')
  } catch (e) {
    console.log('Are you kidding me?')
  }
})()

Also I tried setting the host header as follows:

await page.setRequestInterception(true)

page.on('request', request => {
  const headers = request.headers()

  headers['host'] = 'www.skillshare.com'

  request.continue({ headers })
})

Headers:

{
  host: 'www.skillshare.com',
  'user-agent': '<looooong string>'
}

entrypoint.sh:

#!/bin/bash

nohup google-chrome \
  --no-sandbox \
  --disable-background-networking \
  --disable-default-apps \
  --disable-dev-shm-usage \
  --disable-extensions \
  --disable-gpu \
  --disable-setuid-sandbox \
  --disable-sync \
  --disable-translate \
  --headless \
  --hide-scrollbars \
  --lang=en_US \
  --metrics-recording-only \
  --mute-audio \
  --no-first-run \
  --safebrowsing-disable-auto-update \
  --ignore-certificate-errors \
  --ignore-ssl-errors \
  --ignore-certificate-errors-spki-list \
  --user-data-dir=/tmp \
  --remote-debugging-port=9222 \
  --remote-debugging-address=0.0.0.0

Note:

I tested with puppeteer.launch() and it worked, but I need it to work using puppeteer.connect(). I can also easily connect by domain, but IP does not work.

Error:

Error: net::ERR_INVALID_ARGUMENT at http://104.20.0.109
    at navigate (/app/node_modules/puppeteer/lib/FrameManager.js:120:37)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async FrameManager.navigateFrame (/app/node_modules/puppeteer/lib/FrameManager.js:94:17)
    at async Frame.goto (/app/node_modules/puppeteer/lib/FrameManager.js:406:12)
    at async Page.goto (/app/node_modules/puppeteer/lib/Page.js:672:12)
  -- ASYNC --
    at Frame.<anonymous> (/app/node_modules/puppeteer/lib/helper.js:111:15)
    at Page.goto (/app/node_modules/puppeteer/lib/Page.js:672:49)
    at Page.<anonymous> (/app/node_modules/puppeteer/lib/helper.js:112:23)
    at c.<anonymous> (/app/build/bundle.min.js:1:3204)
    at /app/build/bundle.min.js:1:1256
    at Object.next (/app/build/bundle.min.js:1:1361)
    at n (/app/build/bundle.min.js:1:107)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)

Related Questions:

Update:

As I understand it, there is no way to set Host, so maybe this question has no answer.

1 Answers

Make sure to launch your chrome instance with --remote-debugging-port=9222 to let puppeteer.connect() connect to the browser.

Edit: I see you are already using this flag.

If it wouldn't solve your problem: consider connecting via websocket endpoint instead of browserURL.

Related