download a list of links with reptitive names with wget

user_4168794 0
Viewed 38

I have a list of links, but there are links with different files with the same name. Here is a snippet of my to_download.txt file:

https://www.url.domain/world/2000/may/15/one
https://www.url.domain/world/2000/nov/07/two
https://www.url.domain/world/2000/nov/17/three
https://www.url.domain/world/2000/apr/17/two
https://www.url.domain/world/2000/feb/13/one
https://www.url.domain/world/2000/jun/26/three
https://www.url.domain/world/2000/apr/25/one

When I use wget -i /to_download.txt there is only one file for the URLs with repetitive file names (one one, one two, one three, etc.)

wget download
2 Answers

Because you are overwriting the files. You can't have two files with the same name. You can create separate folder for each month or whatever is the pattern.

user_137977860

This is what I did finally. Suppose all links are in a list named l:

for url in l:
    n = url.split('/')
    name = n[-1] + '_' + n[-2] + '_' + n[-3] + '_' + n[-4]
    os.system('wget ' + url + ' -O ' + name)

I don't think it is the best solution, but it solved my problem.

user_41687940
Related