I am trying to join two datasets that share the same index by using:
merged_data = df1.join(df2)
However, the kernel keeps dying. I've attempted restarting my notebook (jupyter lab) but I think it's related to one of the data frames being about 2GB...
About df1
<class 'pandas.core.frame.DataFrame'>
Index: 97812 entries, XXXX to XXXX
Data columns (total 19 columns):
dtypes: float64(2), int64(3), object(14)
memory usage: 14.9+ MB
about df2
<class 'pandas.core.frame.DataFrame'>
Index: 13888745 entries, XXXX to XXXX
Data columns (total 18 columns):
dtypes: int64(16), object(2)
memory usage: 2.0+ GB
How can I make this work?
I do need all the entries and columns. The dataframes don't share columns in common besides the index.
If it is worth knowing... I am using a MacBook Pro (Early 2015) with 2.7 GHz Dual-Core Intel Core i5 (processor) and 8 GB 1867 MHz DDR3 (memory)