Interpolation of 90 days of missing data

Viewed 52

I have an hourly data. In some days, I only have 2 or 3 observations. So I resampled the data so that I can have an observation for each hour of the days. Then there is this big gap of about 90 days, which is the gap that I am most interested in interpolating.

I resampled the data using the following code: data = data.resample('h').sum()

and this is what I get after running the code:

    power
date                         

2018-01-08 15:00:00  0.226917

2018-01-08 16:00:00  0.000000

2018-01-08 17:00:00  0.000000

2018-01-08 18:00:00  0.000000

2018-01-08 19:00:00  0.207167

2018-01-08 20:00:00  0.000000


2018-01-08 21:00:00  0.000000

2018-01-08 22:00:00  0.000000

2018-01-08 23:00:00  0.000000
  • Basically I get an observation for every hour. And since these observations do not exist, a value of "zero" is assigned to them.

Then I replace the zeroes with NaN:

data.replace(0, np.NaN, inplace=True)

and this is what I get after running the code:

power
date    
2018-01-08 15:00:00 0.226917

2018-01-08 16:00:00 NaN

2018-01-08 17:00:00 NaN

2018-01-08 18:00:00 NaN

2018-01-08 19:00:00 0.207167

2018-01-08 20:00:00 NaN

2018-01-08 21:00:00 NaN

2018-01-08 22:00:00 NaN

2018-01-08 23:00:00 NaN

2018-01-09 00:00:00 NaN

2018-01-09 01:00:00 NaN

After this, I interpolated over the NaN values:

data.interpolate()

and this is what I got after running the code:

    power
date                         
2018-01-08 15:00:00  0.226917

2018-01-08 16:00:00  0.205677

2018-01-08 17:00:00  0.205725

2018-01-08 18:00:00  0.205774


2018-01-08 19:00:00  0.207167

...                       ...

2019-05-14 11:00:00  0.177042

2019-05-14 12:00:00  0.247216

2019-05-14 13:00:00  0.247174

2019-05-14 14:00:00  0.247133

2019-05-14 15:00:00  0.104208

Up to here I think that everything worked fine. However, when I plot the interpolated data, this is what I get using the code below:

inte.plot() 
pyplot.show()

enter image description here

When I plot the data with the following code:

fig = plt.figure(figsize=(18,16))
ax = fig.add_subplot(5,1,1)
ax.plot(data,linewidth=1)
ax.set_title('Load power (hourly data)')
ax.tick_params(axis='both', which='major')
plt.xlabel('Time (Hour)')
plt.ylabel('Load Energy (kW)')

that's what I get: enter image description here

My question is: why isn't the interpolated data showing in the graphs and when it does it shows as a straight line? It should only be a straight line if the values were constant but they are not.

Also, I tried to interpolate over the zeroes without replacing them with NaN but still no luck.

1 Answers

When you use the .interpolate method, it does not modify the original variable. Try this instead:

data = data.interpolate()
Related