pandas loading compressed csv from html response directly

EddyG 0
Viewed 25

Currently I have a way to save the file from the html response:

_header={}
_header['Prefer']='respond-async'
_header['Content-Type']='application/json; odata.metadata=minimal'
_header['Accept-Charset']='UTF-8'
_header['Authorization']='Token'+token

resp=get(_getResultURL,headers=_header,stream=True)
 with open(outputfilepath, 'wb') as f:
     f.write(resp.raw.read())

This writes a compressed csv to outputfilepath adn can be loaded like this:

df=pd.read_csv(outputfilepath,compression='gzip')

I tried loading it directly in two ways, both unsuccessful:

pd.read_csv(resp.raw.read(),compression='gzip')
pd.read_csv(io.BytesIO(resp.raw.read()),compression='gzip')

How can this be done in one go?

csv pandas html
1 Answers

The key was to get the content of the response instead of reading the raw output

df = pd.read_csv(io.BytesIO(resp.content))
EddyG0
Related