How do I capture a specific AJAX (XHR) event in the Chrome Developer Tools?

user_2766558 0
Viewed 225

I'm trying to scrape some information from the United States Postal Service's Every Door Direct Mail service. The service is straightforward. On the USPS EDDM website, a user enters a ZIP code, e.g. 10030, clicks the search button, and that displays a map. That call is easy to capture in a browser developer console. That's just a GET request like this:

https://gis.usps.com/arcgis/rest/services/EDDM/selectZIP/GPServer/routes/execute?f=json&env:outSR=102100&ZIP=10030&Rte_Box=R&UserName=EDDM

The problem, though, is trying to capture the table or the textual information displayed on the right of the page. The "Show Table" option:

enter image description here

displays output like this, at which point you can click "Select All (10 routes)" and it will show you something like on the right side of the page:

enter image description here

How do I capture this set of Javascript "button-click" requests? For now I'm just trying to capture these requests so I can replicate them programmatically (in Python, but that's not strictly relevant here).

enter image description here

google-chrome post ajax
1 Answers

It looks like you need to make two GET requests to get all of the data. The second request is necessary to obtain the total number of PO Boxes. Perhaps they can be combined into one, I didn't dig into it too much.

  1. https://gis.usps.com/arcgis/rest/services/EDDM/selectZIP/GPServer/routes/execute?f=json&env%3AoutSR=102100&ZIP=10030&Rte_Box=R&UserName=EDDM
  2. https://gis.usps.com/arcgis/rest/services/EDDM/selectZIP/GPServer/boxes/execute?f=json&env%3AoutSR=102100&ZIP=10030&Rte_Box=B&UserName=EDDM&extent=%7B%22xmin%22%3A-15628202.596646052%2C%22ymin%22%3A749662.6717286934%2C%22xmax%22%3A-6235620.560966092%2C%22ymax%22%3A7070087.666571667%2C%22spatialReference%22%3A%7B%22wkid%22%3A102100%7D%7D

I know you're working in Python but since you're playing in Chrome DevTools, here's a JavaScript example of how to obtain the total number of delivery addresses:

let promises = [
    fetch('https://gis.usps.com/arcgis/rest/services/EDDM/selectZIP/GPServer/routes/execute?f=json&env%3AoutSR=102100&ZIP=10030&Rte_Box=R&UserName=EDDM'),
    fetch('https://gis.usps.com/arcgis/rest/services/EDDM/selectZIP/GPServer/boxes/execute?f=json&env%3AoutSR=102100&ZIP=10030&Rte_Box=B&UserName=EDDM&extent=%7B%22xmin%22%3A-15628202.596646052%2C%22ymin%22%3A749662.6717286934%2C%22xmax%22%3A-6235620.560966092%2C%22ymax%22%3A7070087.666571667%2C%22spatialReference%22%3A%7B%22wkid%22%3A102100%7D%7D')
];

Promise.all(promises)
    .then(async res => {
        if (res[0].ok && res[1].ok) {
            const first = await res[0].json();
            const second = await res[1].json();

            const results = [...first.results[0].value.features, ...second.results[0].value.features];
            
            console.log("Total Residential Addresses:", results.reduce((acc, row) => acc + row.attributes.RES_CNT, 0));
            console.log("Total Business Addresses:", results.reduce((acc, row) => acc + row.attributes.BUS_CNT, 0));
            console.log("Total Delivery Addresses:", results.reduce((acc, row) => acc + row.attributes.TOT_CNT, 0));
        }
    });

Updated to show how to retrieve residential and business addresses separately.

user_80615550
Related