Concatenate two DataFrames in Pandas row by row horizontally for very large dataframes

Viewed 19

I have two dataframes which are obtained after a lot of pre-processing. I want to combine/merge them row by row and write them directly to a csv file. As the dataframes are huge so using concat and merge options of Pandas results in MemoryError.

Edit: The solutions available on Stackoverflow using merge and concat functions of the pandas are not applicable and results in MemoryError. The dataframes are too large.

Here is an MWE:

DF1

ID F1 F2
1 0 1
2 1 1

DF2

ID F3 F4 Label
1 A B L1
2 C D L2

Resultant CSV File

ID,F1,F2,ID,F3,F4,Label <br>
1,0,1,1,A,B,L1 <br>
2,1,1,2,C,D,L2

Note: Both the dataframes have the same number of rows.

0 Answers
Related