Pandas keep original placement of values

user_9961101 0
Viewed 65

I am removing duplicate data in my rows from my dataframe by using the following:

df = pd.DataFrame(list(map(pd.unique, df.values)))

However, this then alters the placement of my data with the original column names and when reading the data im reading it wrong due to it all being moved. Instead deleting the data is there a way of putting a value in there such as "none" to make it easier to filter through.

Dataframe:

T1583.005   T1583.006   Resource Development    Resource Development    T1583.001   T1583.002
T1584.005   T1584.006   Resource Development    Resource Development    T1584.005   T1584.002

Currently:

T1583.005   T1583.006   Resource Development    T1583.001   T1583.002
T1584.005   T1584.006   Resource Development    T1584.002

Expected:

T1583.005   T1583.006   Resource Development    None    T1583.001   T1583.002
T1584.005   T1584.006   Resource Development    None    None        T1584.002

This way the data stays in its original position and can be read easier.

pandas dataframe duplicates python
1 Answers

Here is one way of doing by masking the duplicated values per row:

df.mask([df.loc[idx].duplicated().tolist() for idx in df.index])

           0          1                     2    3          4          5
0  T1583.005  T1583.006  Resource Development  NaN  T1583.001  T1583.002
1  T1584.005  T1584.006  Resource Development  NaN        NaN  T1584.002
user_128331660
Related