Replace specific row-wise duplicate cells in selected columns without dropping rows

user_1996842 0
Viewed 44

How can I replace specific row-wise duplicate cells in selected columns without dropping rows (preferably without looping through the rows)?

Basically, I want to keep the first value and replace the remaining duplicates in a row with NAN.

For example:

df_example = pd.DataFrame({'A':['a' , 'b', 'c'], 'B':['a', 'f', 'c'],'C':[1,2,3]})
df_example.head()

Original:

    A   B   C
0   a   a   1
1   b   f   2
2   c   c   3

Expected output:

    A   B   C
0   a   nan 1
1   b   f   2
2   c   nan 3

A bit more complicated example is as follows:

Original:

    A   B   C D 
0   a   1   a 1
1   b   2   f 5
2   c   3   c 3

Expected output:

    A   B   C D 
0   a   1   nan nan
1   b   2   f 5
2   c   3   nan nan
pandas-1.0 pandas python
1 Answers

Use DataFrame.mask with Series.duplicated per rows in DataFrame.apply:

df_example = df_example.mask(df_example.apply(lambda x: x.duplicated(), axis=1))
print (df_example)
   A    B  C
0  a  NaN  1
1  b    f  2
2  c  NaN  3

With new data:

df_example = df_example.mask(df_example.apply(lambda x: x.duplicated(), axis=1))
print (df_example)
   A  B    C    D
0  a  1  NaN  NaN
1  b  2    f  5.0
2  c  3  NaN  NaN
user_29010020
Related