How to combine two dataframes with different shapes on multiple column values

Viewed 196

Thank you in advance for your time and effort to answer. I'm trying to combine two dataframes (one containing current data, the other containing future prediction data) with different shapes.

For example,

df1 = ['Date', 'ColA', 'ColB', 'ColC', 'GroupNumber']

df2 = ['Date', 'Col1', 'Col2', 'Col3', 'GroupNumber']

where df1 has a shape of (3000, 5) and df2 has a shape of (5000, 5).

I want to combine these two dataframes on the same 'Date' and 'GroupNumber' column values.

In the end, it would look something like :

final_df = ['Date', 'ColA', 'ColB', 'ColC','Col1', 'Col2', 'Col3', 'GroupNumber'] with the shape of (5000, 8)

Since df1 is smaller than df2, some rows will be empty when combined.

Of course, these shape is just an arbitrary number I came up with as an example. I've tried

final_df = df1.join(df2, lsuffix='_actual', rsuffix='_prediction', on='GroupNumber','Date') 

But this didn't exactly work. If anyone has a suggestion, i'd really appreciate it. Thank you!

1 Answers

You can use pandas merge to perform a left join:

import pandas as pd
final_df = pd.merge(df1,df2,on=['GroupNumber','Date'],how='left')
Related