Can you get all the network traffic (requests and responses) of the browser using puppeteer?

Viewed 943

I am attempting to grab relevant elements on a page and then click each of them in order to capture the redirects in between and the final landing url.

const page = await browser.newPage();
const url = "someurl";

await page.goto(url, {
    waitUntil: 'networkidle0',
    timeout: 30000
});

//Grab the clickable elements
let anchors, buttons, elements;
anchors = await page.$$('a');
buttons = await page.$$('button');
elements = anchors.concat(buttons);

// Click each element and wait for navigation
for (var i = elements.length - 1; i >= 0; i--) {

    // targetcreated event fires once new page is opened
    const newPagePromise = new Promise(fulfill => browser.once('targetcreated', target => fulfill(target.page())));

    elements[i].click({button: 'middle'});

    // wait for newPage to be created from above promise
    const newPage = await newPagePromise;
    if(newPage != null){
        newPage.on('request', interceptedRequest => {
            console.log(`InterceptedRequest.url: ${interceptedRequest.url}`);
        });
    }
}

The page's initial redirects happen before I can set the page to save the redirects via the page.on('request'....) function.

Is there a way to globally set any new pages to have a page.on like above or a way to manage all the responses for the browser?

Big thanks to Puppeteer dev, great tool.

0 Answers
Related