Pandas: How to append a single row to a pickled and zipped dataframe efficiently?

Viewed 19

I'm in a situation where I have to add a single row to the end of dataframe very frequently. Initially I used plain text .csv files and therefore appending a line at the end of the file was trivial and didn't require loading the dataframe to RAM.

line_to_add = '1,2,3\n'
with open('path/to/file.csv', 'a') as file_handle:
    file_handle.write(line_to_add)

For memory disk reasons I would like to save my dataframe as a pickled+zipped file, but if I do that I loose the ability to easily append to the end of the file. Is this doable without having to load the dataframe into RAM every time?

0 Answers
Related