Drop rows in pandas where all values are the same

user_9749124 0
Viewed 83

I want to drop rows where all values are the same. But, I want solution that can apply to 1,2,3,4...n columns.

df = pd.DataFrame({"Col1":[1,4,2,5,1,4],
                  "Col2":[4,5,2,2,3,4],
                  "Col3":[5,1,2,5,1,4],
                  "Col4":[3,1,2,4,2,4]})

print(df)

   Col1  Col2  Col3  Col4
0     1     4     5     3
1     4     5     1     1
2     2     2     2     2    # delete
3     5     2     5     4
4     1     3     1     2
5     4     4     4     4    # delete

This is the example of 4 columns, but I do not know how to find solution for this. So in this case, rows 2 and 5 should be dropped.

pandas python
3 Answers

nunique

df[df.nunique(axis=1) > 1]

   Col1  Col2  Col3  Col4
0     1     4     5     3
1     4     5     1     1
3     5     2     5     4
4     1     3     1     2

nunique(axis=1) tells you the number of unique values in a row:

df.nunique(axis=1)

0    4
1    3
2    1
3    3
4    3
5    1
dtype: int64

From here perform a comparison,

df.nunique(axis=1) > 1 

0     True
1     True
2    False
3     True
4     True
5    False
dtype: bool

to find the rows that satisfy your condition, then boolean index into df.

user_49090870

Something different like std, standard deviation equal to 0 mean all value is same 

df[df.std(1)!=0]
   Col1  Col2  Col3  Col4
0     1     4     5     3
1     4     5     1     1
3     5     2     5     4
4     1     3     1     2
user_79645270

A lot of puritan/performance pandas enthusiastics are gonna kill me, but:

from pandas import DataFrame
df = DataFrame({"Col1":[1,4,2,5,1,4],
                  "Col2":[4,5,2,2,3,4],
                  "Col3":[5,1,2,5,1,4],
                  "Col4":[3,1,2,4,2,4]})
df_copy = df
for i,j in df_copy.iterrows():
    n_of_unique_elements = len(list(set(df_copy.iloc[i].values)))
    if n_of_unique_elements<=1:
        df = df.drop([i])
print(df)
user_119962660
Related