How to find duplicates row wise in multiple columns in Python

Asif Sayed 0
Viewed 44

I've dataframe with unknown columns. I need to find duplicates in column which start with OEInterestTraceData and only if they are more than 1 in column. Also need to assign 1 new variable/flag if they are duplicate in row then 1 else 0.

W_Interest = loaddata1.loc[:, loaddata1.columns.str.contains('OEInterestTraceData')]

if len(W_Interest.columns) > 1:
    if W_Interest[W_Interest.duplicated(keep=False)]:
        W_Interest[['dummy']] = 1
    else:
        W_Interest[['dummy']] = 0
else:
    W_Interest[['dummy']] = 0

right now it is giving me below error:

ValueError: The truth value of a DataFrame is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

Output should be like this:
enter image description here

Please help out if you have any better solutions.

pandas numpy python
1 Answers

You could create a column of zeros first, then mark those duplicated rows with value 1, for example:

W_Interest = loaddata1.loc[:, loaddata1.columns.str.contains('OEInterestTraceData')]

if len(W_Interest.columns) > 1:
    W_Interest['dummy'] = 0    #assign zero to all rows first
    W_Interest.loc[W_Interest.duplicated(keep=False), 'dummy'] = 1    #then for those rows that are duplicated assign 1 
else:
    W_Interest['dummy'] = 0
perpetualstudent0
Related