I am trying to extract a webpage off of the internet, but on some pages (e.g. Tesco's website) I get warning and then it just hangs.
Warning is
InsecureRequestWarning: Unverified HTTPS request is being made to host 'www.tesco.com'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
I am using requests to make the connection and get the page.
Minimum working code:
import requests
def extract_page(url):
page = requests.get(url, verify=False)
print(f'status code : {page.status_code}')
if page.status_code != 200:
print(f"URL '{url} returned status code {page.status_code}")
return
extract_page('https://www.tesco.com/')
Any guidance on what I am missing would be greatly appreciated, especially as some sites such as Wikipedia work just fine and I couldn't see any other pages that addresses it.