Compare two dataframes and add new values in second dataframe to the first data-frame

user_15422975 0
Viewed 346

I have two dataframes with the same headers

df1\

      **Date  prix moyen    mini   maxi H-Value C-Value**

0   17/09/20     8     6       9    122 2110122\
1   15/09/20     8     6       9    122 2110122\
2   10/09/20     8     6       9    122 2110122

and

df2

      **Date     prix   moyen    mini   maxi    H-Value C-Value**\
1   07/09/17     1.80    1.50    2.00   170     3360170\
1   17/09/20     8.00    6.00    9.00   122     2110122\
2   17/09/20     9.00    8.00   12.00   122     2150122\
3   17/09/20    10.00    8.00   12.00   122     14210122

I want to compare the two dataframes alone 3 parameters (Date, H-Value and C-Value), identify the new values present in df2 (values which do not occur in df1) and then append them in df1.

I am using

df_unique = df2[~(df2['Date'].isin(df1['Date']) & df2['H-Value'].isin(df1['H-Value']) & df2['C-Value'].isin(df1['C-Value']) )].dropna().reset_index(drop=True)

and it is not working in identifying the new values in df2. The resulting table only identifies some values and not others.

Where am I going wrong?

pandas dataframe python
3 Answers

What is your question?

In [4]: df2[~(df2['Date'].isin(df1['Date']) & df2['H-Value'].isin(df1['H-Value']
   ...: ) & df2['C-Value'].isin(df1['C-Value']) )].dropna().reset_index(drop=Tru
   ...: e)
Out[4]: 
   Date      prix  moyen  mini  maxi  H-Value   C-Value
0     1  07/09/17    1.8   1.5   2.0      170   3360170
1     2  17/09/20    9.0   8.0  12.0      122   2150122
2     3  17/09/20   10.0   8.0  12.0      122  14210122

These are all rows in df2 that are not present in df1. Looks good to me...

user_65646200

I was actually able to solve the problem. The issue was not the command being used to compare the two datasets but rather the fact that one of the columns in df2 had a data format different from the same column in df1, rendering a direct comparison not possible.

user_154229750

Here's what I try

df1 = pd.concat([df1, df2[~df2.set_index(['Date', 'H-Value', 'C-Value']).index.isin(df1.set_index(['Date', 'H-Value', 'C-Value']).index)]])
user_134545350
Related