the memory usage of my .csv file in pandas is bigger than my RAM. how is this possible?

Viewed 332

Please help me understand. I have 4 GB of RAM. I read in a full csv file that is 2.1GB large in pandas using

data = pd.read_csv('~/data.csv')

Now when I check the memory usage of the file with the following code:

data.memory_usage(deep=True).sum()

It returns 6012181982 bytes or 6GB. How is it possible that the memory usage is larger than my actual RAM and I won't get any memory errors?

0 Answers
Related