I'd like to scrape all the comments on some youtube videos using the youtube API, but I'm having trouble exceeding the 100 maxresult. This is my code:
video_response = youtube.commentThreads().list(part='snippet, replies',videoId=video_id, maxResults=100, pageToken=pageToken if pageToken != "" else "").execute()
for item in video_response['items']:
comment = item['snippet']['topLevelComment']['snippet']['textDisplay']
date = item['snippet']['topLevelComment']['snippet']['publishedAt']
count = item['snippet']['totalReplyCount']
if count > 0:
parent_id = item['snippet']['topLevelComment']['id']
response = youtube.comments().list(part="snippet", parentId=parent_id).execute()
for x in response['items']:
print(x['snippet']['textDisplay'])
if "nextPageToken" in video_response:
pageToken = video_response.get('nextPageToken')
The comments sometimes are over 4k and no metter what I always get only 100 results. What am I doing wrong? Thakn you all