Going from monthly average dataframe to an interpolated daily timeseries

user_10382580 0
Viewed 476

I am interested in taking average monthly values, for each month, and set the monthly average values to be the value on the 15th day of each month (within a daily timeseries).

I start with the following (these are the monthly average values I am given):

m_avg = pd.DataFrame({'Month': ['1.527013956', '1.899169054', '1.669356146','1.44920871', '1.188557788', '1.017035727', '0.950243755', '1.022453993', '1.203913739', '1.369545041','1.441827406','1.48621651']

EDIT: I added one more value to the dataframe so that there are now 12 values.

Next, I want to put each of these monthly values on the 15th day (within each month) for the following time period:

ts = pd.date_range(start='1/1/1950', end='12/31/1999', freq='D')

I know how to pull out the date on 15th day of an already existing daily timeseries by using:

df= df.loc[(df.index.day==15)] # Where df is any daily timeseries

Lastly, I know how to interpolate the values once I have the average monthly values on the 15th day of each month, using:

df.loc[:, ['Col1']] = df.loc[:, ['Col1']].interpolate(method='linear', limit_direction='both', limit=100)

How do I get from the monthly DataFrame to an interpolated daily DataFrame, where I linearly interpolate between the 15th day of each month, which is the monthly value of my original DataFrame by construction?

EDIT:

Your suggestion to use np.tile() was good, but I ended up needing to do this for multiple columns. Instead of np.tile, I used:

index = pd.date_range(start='1/1/1950', end='12/31/1999', freq='MS')
m_avg = pd.concat([month]*49,axis=0).set_index(index)

There may be a better solution out there, but this is working for my needs so far.

pandas-groupby pandas dataframe interpolation python-3.x
1 Answers

Here is one way to do it:

import pandas as pd
import numpy as np

# monthly averages, note these should be cast to float 
month = np.array(['1.527013956', '1.899169054', '1.669356146', 
                  '1.44920871',  '1.188557788', '1.017035727',
                  '0.950243755', '1.022453993', '1.203913739', 
                  '1.369545041', '1.441827406', '1.48621651'], dtype='float')

# expand this to 51 years, with the same monthly averages repeating each year
# (obviously not very efficient, probably there are better ways to attack the problem, 
# but this was the question)
month = np.tile(month, 51)

# create DataFrame with these values
m_avg = pd.DataFrame({'Month': month})

# set the date index to the desired time period
m_avg.index = pd.date_range(start='1/1/1950', end='12/1/2000', freq='MS')

# shift the index by 14 days to get the 15th of each month
m_avg = m_avg.tshift(14, freq='D')

# expand the index to daily frequency
daily = m_avg.asfreq(freq='D')

# interpolate (linearly) the missing values
daily = daily.interpolate()

# show result
display(daily)

Output:

            Month
1950-01-15  1.527014
1950-01-16  1.539019
1950-01-17  1.551024
1950-01-18  1.563029
1950-01-19  1.575034
...         ...
2000-12-11  1.480298
2000-12-12  1.481778
2000-12-13  1.483257
2000-12-14  1.484737
2000-12-15  1.486217

18598 rows × 1 columns
user_130141720
Related