I just wrote a script that scrape all youtube links in a given page (this one : https://coreyms.com/) and open them in a browser (this has no particular use beside learning how to do it).
from requests_html import HTML, HTMLSession
import re
import webbrowser
url = input("website url to launch all embeded videos")
if url == '':
url = 'https://coreyms.com/'
sess = HTMLSession()
r = sess.get(url)
whole_file = r.text
pattern = re.compile(r'https:\/\/www\.youtube\.com\/embed\/(.+)[\?"]')
video_ids = pattern.findall(whole_file)
The problem is that it only works with that specific website (because I used the HTML source of it in order to know what my script must search).
I managed to make the script works with this as well
https://www.udiscovermusic.com/stories/best-heavy-metal-songs/
But with another one :
https://www.musicgrotto.com/best-classic-rock-songs/
It does not work again. I think this is not the good approach. \
How can I generalise my script in order to make it work with any website ? Is there a common tread I could use ?
I don't know much of HTML and web developpement. (that's exactly why I want to do this)
Thanks