How to extract videos info of a huge channel using python youtube-dl?

user_991395 0
Viewed 244

I'm using the following code to extract videos info (without downloading):

import youtube_dl

with youtube_dl.YoutubeDL({}) as ydl:
    result = ydl.extract_info(
        'https://www.youtube.com/channel/UCIG1k8umaCIIrujZPzZPIMA',
        download=False
    )

The above code is working for a small channel (100 videos) but for a huge channel (1000 videos) it shows DownloadError: ERROR: Unable to download webpage: HTTP Error 429: Too Many Requests after 200 videos.

I think it should wait a while every 100 videos for example.

How to solve it?

youtube-dl youtube python
1 Answers

dirty hack

modify the logic here (youtube-dl) or here (yt-dlp) from

if self._num_downloads exceeds max_downloads, then
    abort

to

if self._num_downloads is divisible by 100, then
    wait for a while

then you

result = ydl.extract_info('https://www.youtube.com/channel/UCIG1k8umaCIIrujZPzZPIMA',
                           download=True)

beware that this will NOT work if you pass download=False

user_82439910
Related