I want to scrape data from Spotify Charts for my Big Data projects, but since Spotify recently changed, it doesn't allow users to see the charts without logging in, my python code didn't work like before, because https://charts.spotify.com/ would redirect me to their home page instead of the link i want to scrape the data. Any ways to get around with this, thanks. This is my python code:
from bs4 import BeautifulSoup
import requests
import urllib.request
from selenium import webdriver
from webdriver_manager.firefox import GeckoDriverManager
###get all content on home_page
url='https://charts.spotify.com/charts/view/regional-global-weekly/latest'
hsession = requests.Session()
hresponse = hsession.get(url)
soup=BeautifulSoup(hresponse.content.decode('utf-8'),"html.parser")
table=soup.find('table', 'Table__TableElement-hzsdh5-0 dKzsET ChartsTable__StyledTable-sc-1ye7294-5 bovxiZ')
links = table.findAll('a')
print(links); ###Error