Replace a row in Pandas DataFrame with 'NaN' based on condition

user_11415831 0
Viewed 1872

I have a Pandas DataFrame called df (378000, 82) and I would like to replace the entire row with NaN based on a specific condition. The condition is for any value in the column df.halon_gas that is >20, I want to replace that entire row with NaN. This is the way I want to filter my data so I don't lose the index values.

Thanks!

pandas dataframe filter python
2 Answers

If you're fine with the rows being gone then I suggest you do this:

df.reset_index(level=0, inplace=True)
df = df[df.halon_gas <= 20]
df.set_index("index", inplace=True)

Whats happening here is the following:

  1. The Index gets reset so you have an extra Column with the Index Values pre Removal.
  2. Only the rows where df.halon_gas <= 20 are kept.
  3. The old Index values are set to be the Index for the DataFrame again.
user_140553260

First of all get all indexes of values, that are below 20

    idx = df[df.halon_gas >= 20].index

Then set the values for all columns and all columns which are below 200 to None

    df.set_value(idx, df.columns , None)

This should write None/Nan in the rows with the value below 20

user_116219510
Related