'Too many requests Error' in a web scraping session

Viewed 34

I'm new in web scraping. I'm trying to download subtitles from https://www.opensubtitles.org with this (for now) rough function:

def subtitle_maker(episode_zip):
    resp=urlopen(episode_zip)
    with zipfile.ZipFile(BytesIO(resp.read())) as z:
        for filename in z.namelist():
                if not os.path.isdir(filename):
                    if filename.endswith('.srt'):
                        with z.open(filename) as f:
                            subtitle=f.read().decode("latin-1")
    return(subtitle)

I have a list of urls like this (https://dl.opensubtitles.org/it/download/sub/8861872) that I use as argument of 'subtitle_maker function' and for each url, with a 'for loop', I try to append the output of that function to the list item, but after a few cycles the program is stopped due to urllib.error.HTTPError: HTTP Error 429: Too Many Requests. How can I manage to avoid this and pursue my goal? Thanks

0 Answers
Related