How to use the requests module to skip connection timeout urls

user_7662166 0
Viewed 312

Hi how can i use the request module to go through a bunch of URLs and if a url in the list takes more time to load or a connection timeout how can i skip that particular url and skip to the next one

def req():
with open('demofile.txt','r') as http:
    for url in http.readlines():
        req = url.strip()
        print(req)
        page=requests.get("http://"+req,verify=False)
        if page.status_code == 400:
            break
        else:
            continue
        time.sleep(1)
python-requests python-3.x python
1 Answers

You can raise exception if there is a timeout and continue on finally block for next request,

import requests
import logging

timeout = 0.00001

try:
    response = requests.get(url="https://google.com", timeout=timeout)
except requests.exceptions.ConnectTimeout as e:
    logging.error("Time out!")
finally:
    # continue request here
    print("hello")


# output,
ERROR:root:Time out!
hello
user_11494230
Related