How to transform missing values ​through a multiplication of a "K" factor starting at the last valid value?

gveronese 0
Viewed 21

My problem is to apply a multiplication factor K=0.5 starting at the first valid number that appears before the missing values ​​"NaN" and keep applying that factor K to the calculated values ​​until the last missing value NaN of the period.

TABLE A:

Bird1  Bird2  Bird3 
    
     100  50      200  
     50   40      100  
     40   40      80  
     NaN  80      200  
     NaN  50      NaN 
     NaN  90      NaN 
     100  12      40

The result should be as per the table below. How to implement this code in python?

TABLE B:

Bird1   Bird2    Bird3 

 100     50      200  
 50      40      100  
 40      40      80  
 20      80      200  
 10      50      100 
 5       90      50 
 100     12      40

Using the df.interpolate() command is not suitable because it also uses the value after the missing values ​​NaN. I would just like a constant K starting and being applied to the first value BEFORE the missing values ​​NaN.

missing-data interpolation pandas numpy python
1 Answers

You can write a custom interpolation function:

def interp(s, K=0.5):
    # group by non-NA then NA
    g = s.groupby(s.notna().cumsum())
    # get decreasing factors
    factor = K**g.cumcount()
    # interpolate
    return g.transform('first').mul(factor)

out = df.apply(interp)
#     df.apply(interp, K=0.5)

output:

   Bird1  Bird2  Bird3
0  100.0   50.0  200.0
1   50.0   40.0  100.0
2   40.0   40.0   80.0
3   20.0   80.0  200.0
4   10.0   50.0  100.0
5    5.0   90.0   50.0
6  100.0   12.0   40.0
mozway0
Related