I'm trying to access a website that verifies I'm not a bot by redirecting after a few seconds. How do I have the requests module wait for a redirect?
Edit: It seems I didn't entirely understand the problem. I liked Kirk Strauser's response, but couldn't find a location header.
I found out that the site was managed with Cloudflare. I tried using cfscrape, which didn't work, and cloudscraper hasn't updated for this captcha version yet. Probably not going to be solved on stackexchange.
The requests module has a parameter for configuring allow_redirects if you need to follow the redirected 300-series response.
https://docs.python-requests.org/en/master/user/quickstart/#redirection-and-history
>>> r = requests.get('http://github.com/', allow_redirects=False)
>>> r.status_code
301
>>> r.history
[]
In that specific case, you can sleep a little while and then manually visit the next location:
import time
import requests
r = requests.get('http://github.com/', allow_redirects=False)
# Don't fetch the next page too quickly
time.sleep(3)
# Now go to the URL that the previous request would have gone to
# if you hadn't asked requests.get not to.
r = requests.get(r.headers['Location'])