Python requests timeout not working properly

user_9437838 0
Viewed 7943

I'm using the following code to check the average response time because i noticed it was very slow:

            t1 = time.time()
            response = requests.get(url, timeout=10, headers=headers)
            t2 = time.time()
            reqtimes += t2 - t1
            reqamount += 1
            print("Average response time:" + str(reqtimes/reqamount))

When i print the average response time after connecting to around 1000 different sites it tells me the average response time is 70 seconds. Why? my timeout is set to 10 !

python-requests request python-3.x python
1 Answers

The timeout is the maximum amount of time to allow for any response from the server*. If the timeout is 10 seconds, and the server returns a 100 byte file, a single byte at a time every 9 seconds, then you'll never time out... but you'll have a very long wait for the response to complete (total elapsed time will be 900 seconds). eg: Even a single request with a timeout of 10 seconds will have an average response time of 15 minutes).

It's worth noting that response objects already have an .elapsed attribute which gives you a timedelta object you can use to avoid using time.time() and performing calculations yourself.

*Paraphrased from http://docs.python-requests.org/en/master/user/quickstart/#timeouts:

timeout is not a time limit on the entire response download; rather, an exception is raised if the server has not issued a response for timeout seconds (more precisely, if no bytes have been received on the underlying socket for timeout seconds). If no timeout is specified explicitly, requests do not time out.

user_12527590
Related