Read DOMStorage / LocalStorage via Chrome DevTools Protocol

user_5506547 0
Viewed 1010

I know the following code snippet allows me to read all current cookies:

let Main = await import('./main/main.js');
await Main.MainImpl.sendOverProtocol('Network.getCookies');

preview Network.getCookies

Now, for testing purposes, I need to get all available LocalStorage entries. I tried a few things but unfortunately I did not get it. Chrome DevTools itself does only query it with the already available securityOrigin variable, and I do not know where it is taken from:

CDP getDOMStorageItems

I also found Page.getFrameTree but there are missing some entries - so I think this does not relate to the entries of LocalStorage:

local storage view

Is there any other method in the docs I am missing?

Edit #1

As mentioned in the comments I got it to work with Page.getResourceTree. Unfortunately the next issue I get is the following:

Protocol error (DOMStorage.getDOMStorageItems): Frame not found for the given security origin.
chrome-devtools-protocol puppeteer google-chrome-devtools
3 Answers

Does this suffice?

'use strict';

const puppeteer = require('puppeteer');

(async function main() {
  try {
    const browser = await puppeteer.launch();
    const [page] = await browser.pages();

    await page.goto('http://example.org/');

    await page.evaluate(() => { window.localStorage.setItem('foo', 42); });

    const cdp = await page.target().createCDPSession();

    const data = await cdp.send('DOMStorage.getDOMStorageItems', {
      storageId: {
        securityOrigin: await page.evaluate(() => window.origin),
        isLocalStorage: true,
      },
    });
    console.log(data);

    await browser.close();
  } catch (err) {
    console.error(err);
  }
})();

Output:

{ entries: [ [ 'foo', '42' ] ] }
user_26255600

I use the following code:

getLocalStorage = async function(page, data = {}) {
  // based on https://stackoverflow.com/a/54355801/1407622
  const client = page._client;
  for (const frame of page.frames()) {
    // it is unclear when the following url values occur:
    // potentially about:blank is the frame before the very first page is browsed
    if(!frame.url().startsWith('http')) continue; // filters chrome-error://, about:blank and empty url

    const securityOrigin = new url.URL(frame.url()).origin;
    const response = await client.send('DOMStorage.getDOMStorageItems',
      { storageId: { isLocalStorage: true, securityOrigin } },
    );
    if (response.entries.length > 0) {
      let entries = {};
      for (const [key,val] of response.entries) {
        entries[key] = {
          value: safeJSONParse(val),
        };
      }
      // console.log(response.entries);
      data[securityOrigin] = Object.assign({}, data[securityOrigin], entries);
    }
  }
  return data;
};

source: https://github.com/EU-EDPS/website-evidence-collector/blob/master/lib/tools.js
original SA source: Get all values from localStorage using Puppeteer

However, sometimes I get the same error: "Frame not found for the given security origin"

user_14076220

Puppeteer may opens a blank chromium tab so you may get bad frame urls like about:blank or empty urls.

I managed to bypass this error by adding two simple check :

      if (!frame.url()) {
        continue;
      }

      if (frame.url() === 'about:blank') {
        continue;
      }

it seems like there are some bad urls too like :

chrome-error://chromewebdata/

user_97260370
Related