Try to Scrape Instacart: Python

user_11369959 0
Viewed 1133

I am having trouble trying to webscrape instacart using the requests library in python. I want to get past the first initial page that asks for the zip code and move onto the email page. Here is my code:

import requests

session = requests.Session()
data = {"address_line_1":"MYZIPCODE"}
header = {'user-agent': "Mozilla/5.0 (iPhone; CPU iPhone OS 5_1 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9B179 Safari/7534.48.3"}
response = requests.post('https://www.instacart.com/', headers=header, data=data)
print(response.text)

I was wondering if you guys could help.

BTW: I am using address_line_1 because that is the name of the input. Thanks

python-requests python
1 Answers

No guarantees, as this answer is the result of some brief experimentation, but I might have found a solution.

I poked around on the website, and discovered that when I reloaded the page, the initial screen asking for your zip code/email was skipped. I then deleted the session cookie, _instacart_session, but left all other cookies intact, and was asked to sign in again.

This suggests that if you sign in manually using your browser, and then save the _instacart_session cookie that is generated, you could bypass the sign-in process. Send the cookie along with the requests you make in your program, and you might be able to skip directly to http://instacart.com/store/NAME_OF_STORE/storefront.

There might even be another cookie that remembers your desired location, thus bypassing that selection screen as well (but I am not sure of this).

Edit: I see delivery times listed in the JSON info for each store, with URLs like https://www.instacart.com/v3/containers/NAME_OF_STORE/next_gen/retailer_information/content/delivery?source=web. You need to navigate to https://www.instacart.com/store/NAME_OF_STORE/info?tab=delivery first to set your cookies, else I get an Unauthorized error message.

Hope this helps!

user_116395330
Related