tqdm not showing progress bar with requests

user_18084172 0
Viewed 301

Why doesn't tqdm work here when combined with requests? When I run the script below, all I get is a blinking cursor, no progress bar. Thank you.

import requests, re
from tqdm import tqdm

link = "https://web.archive.org/cdx/search/cdx?url=twitter.com/realdonaldtrump/status&matchType=prefix&filter=statuscode:200"
flamingo = []

y = requests.get(link).text
urls = re.findall(r'http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+', y)

for url in tqdm(urls):
    flamingo.append(f"{url}")
tqdm python-requests request python
1 Answers

I tried your code and it works fine if you just add a limit argument to your URL.

link = "https://web.archive.org/cdx/search/cdx?url=twitter.com/realdonaldtrump/status&matchType=prefix&filter=statuscode:200&limit=10"

The list of tweets you get without limit is very, very long. I assume that your code doesn't reach your loop before timing out. (This at least was the case when I tested your script)

user_71170030
Related