Im trying to read a google sheet as a dataframe using this code:
link = f'https://docs.google.com/spreadsheets/d/{google_id}/export?format=csv'
r = requests.get(link)
data = r.content
df = pd.read_csv(StringIO(data.decode('utf-8')), skiprows=4, na_values=['NaN'], header=None, names=header_list, index_col=False)
It was working okay, until I found out that some columns have commas in their values and the data frame is detecting it as new column. I tried to use another separator but it was not working. Any idea how to solve this?