How to replace fillna for categories with column names?

user_11053801 0
Viewed 1547

I have a pandas data frame, I want to fill missing categories with 'colname_miss' string.

def FillCatMissing(df):
    cols = ['A','B','C']
    df[cols] = df[cols].fillna('miss')
    return df

This fills all missing categories with string 'miss', I need to be like 'A_miss' for column A, 'B_miss' for column B....

pandas python
3 Answers
for col in df.columns:
    df[col].fillna(col+'_miss', inplace=True)
user_87924510

Why not apply with replace:

def FillCatMissing(df):
    cols = ['A','B','C']
    df[cols] = df[cols].apply(lambda x: x.replace(np.nan, x.name + '_miss'))
    return df
user_87083640

Setup

df = pd.DataFrame(dict(A=['a', None], B=[None, 'b'], C=[None, None]))

df

      A     B     C
0     a  None  None
1  None     b  None

Pass a dict to fillna

Pandas has a specific way to handle this problem. Instead of looping over columns and filling each one individually, you can pass a dictionary to the fillna method that defines what to replace null values with for each column. In other words, this is the way it should be done.

In this case, you want the dictionary's keys to match the column names to be filled.

df.fillna({k: f'{k}_miss' for k in df})  # This is the answer you are looking for

        A       B       C
0       a  B_miss  C_miss
1  A_miss       b  C_miss

We could've left it to just the columns 'A' and 'B'

df.fillna({k: f'{k}_miss' for k in ['A', 'B']})

        A       B     C
0       a  B_miss  None
1  A_miss       b  None

And this leaves column 'C' alone.

Lastly, this produces a copy with the results rather than mutating the existing dataframe. If you want to overwrite the existing dataframe then just assign to the the same name

df = df.fillna({k: f'{k}_miss' for k in df})

And though I clearly like the other answer better, this is another way to do it.

df.fillna(df.columns.to_series().add('_miss'))

        A       B       C
0       a  B_miss  C_miss
1  A_miss       b  C_miss
user_23366540
Related