Drop all rows from DataFrame in which all values, except a specific one, is NaN

Admin 1
Viewed 1765

I have a very large DataFrame with many columns (almost 300). I would like to remove all rows in which the values of all columns, except a column called 'Country' is NaN.

dropna can remove rows in which all or some values are NaN. But what Is an efficient way to do it if there's a column you want to exclude from the process?

pandas python
3 Answers

Try:

mask = df.drop("Country",axis=1).isna().all(1) & df['Country'].notna()
out = df[~mask]
user_98406370

You can use filter to exclude the column Country:

df = pd.DataFrame({"A":[nan, 2, 3,], "Countryside":[nan, 4, 6], "Country":list("ABC")})

     A  Countryside Country
0  NaN          NaN       A
1  2.0          4.0       B
2  3.0          6.0       C

print (df[~df.filter(regex=r"^(?!Country\b)").isnull().all(1)])

     A  Countryside Country
1  2.0          4.0       B
2  3.0          6.0       C
user_92844230

This should work:

def remove_if_eq(v,dataframe,keys):
    for i in df.index:
        delete = True
        for j in df.columns:
            if df[j][i] != v:
                delete = False
                break
        if delete:dataframe = dataframe.drop(i)
    return(dataframe)

v is the value dataframe is the dataframe keys are the names of the colums you want searched

user_124692750
Related