Python how to use Threading in multi download

user_12395277 0
Viewed 72

I am use threading to do parallel download now i have a url_list img_list i want to download it in 2 thread so def two download.

i put half into download1 and half in download2 so it will speed up to complete, but finally when i run the scripts my download still in serial,i don't know why, how can i modify my script?

here's the code:

import requests,threading
img_list=[...]
num=len(img_list)
def download_1(img_list):
    n=0
    for i in img_list:
        n+=1
        with open('./img/'+str(n)+'.jpg','wb')as f:
            f.write(requests.get(i).content)
            print(str(n)+"download1 complete")
def download_2(img_list):
    n=len(img_list)
    for i in img_list:
        n+=1
        with open('./img/'+str(n)+'.jpg','wb')as f:
            f.write(requests.get(i).content)
            print(str(n)+"download2 complete")
thread_1 = threading.Thread(target=download_1(img_list[:int(num/2)]))
thread_2 = threading.Thread(target=download_2(img_list[int(num/2):]))
thread_1.start()
thread_2.start()
python-requests python
2 Answers

In this line

threading.Thread(target=download_1(img_list[:int(num/2)]))

you call download_1(...) and pass the result (null) to thread. That's why it runs serially. Instead you want to pass download_1 function itself (not the result of calling it) to the thread. Like this:

threading.Thread(target=download_1, args=(img_list[:int(num/2)],))

Do it in both places.

Side note: you should t.join() both threads at the end.

user_6455510

You are calling both functions at the time of creating threads. So, threads are passed null and, therefore, do nothing. You should change the code like this:

thread_1 = threading.Thread(target=download_1, args=(img_list[:int(num/2)]))
thread_2 = threading.Thread(target=download_2, args=(img_list[int(num/2):]))

thread_1.start()
thread_2.start()
user_113958610
Related