How to fill NANs with values tending to zero until the next valid value?

user_8776627 0
Viewed 43

While resampling a dataframe (df) as:

df = pd.DataFrame.from_dict({'2021-03-02': 442,
'2021-03-04': 520,
'2021-03-09': 390,
'2021-03-11': 442,
'2021-03-16': 520,
'2021-03-23': 520,
'2021-03-25': 520,
'2021-03-26': 442,}, orient='index',)

df.index = pd.to_datetime(df.index)
df = df.resample('30Min').asfreq()

How do I fill the NANs with values that linearly tend to zero from their predecessor? (a graphic would be looking like a saw)

Are there any built in methods for this operation or a custom method needs to be used in conjuncture with .apply()?

Thank you for your time.

pandas python-3.x
2 Answers

There's no built-in function for this. You can create one quickly like this:

# group of rows starting with non-nan
groups = df[0].groupby(df[0].notnull().cumsum())

# output
out = df[0].ffill().mul(1-groups.cumcount()/ groups.transform('size'))

# plot
out.plot()

And you get:

enter image description here

user_42384080

Another option is to fill the nan just before a non-nan value with 0 using notnull and shift, then interpolate.

df.loc[df[0].notnull().shift(-1, fill_value=False), 0] = 0
df[0] = df[0].interpolate()
user_92747320
Related