fill in the data NaN with the data that is above it

user_14936778 0
Viewed 52

I have dataframe with the data below : ii want filling NaN in the data with the data on above data.

   a   b   c
0   1   1   1
1   2   2   2
2   3   3   3
3   NaN NaN 4
4   NaN NaN 5

I want to push the data to NaN to make it look like this :

    a   b   c
0   NaN NaN 1
1   NaN NaN 2
2   1   1   3
3   2   2   4
4   3   3   5
pandas dataframe python
1 Answers

You can create a null values mask, then concatenate the non-null and null rows:

>>> nullMask = df.isnull().sum(axis=1).astype(bool)
>>> pd.concat([df[nullMask], df[~nullMask]])

OUTPUT:

    a   b   c
1           
4   NaN NaN 4
5   NaN NaN 5
2   1.0 1.0 1
2   2.0 2.0 2
3   3.0 3.0 3

Since, you don't want the values in non-null columns i.e. c in the sample data to be reordered, you need two maskings: one for columns to know which columns have null values, and one for indices in-order to shift the null values in the columns to the begining:

nullCols = df.isna().sum()
nullCols = nullCols[nullCols.gt(0)].index.tolist() 
nullMask=df[cols].isnull().sum(axis=1).astype(bool)
out=pd.concat([df[nullCols][nullMask], df[nullCols][~nullMask]], ignore_index=True)
remainingCols = df.columns[~df.columns.isin(nullCols)].tolist()
out[remainingCols] = df[remainingCols].values

OUTPUT:


    a   b   c
0   NaN NaN 1
1   NaN NaN 2
2   1.0 1.0 3
3   2.0 2.0 4
4   3.0 3.0 5
user_91363480
Related