FutureWarning: .loc or [] with missing label, suggests to use .reindex()

user_9230374 0
Viewed 3291

My input dataframe has Date in quarter format and Rate. There are missing quarter in Date. Now I am trying to compute the Update based on logic that if 3rd last quarter row is available then pick this Rate otherwise pick the last row Rate. My current code looks like below:

# Create dataframe
df1 = pd.DataFrame([[1, '2015Q3'], [2, '2015Q4'], 
                    [6, '2017Q1'], [7, '2017Q4'], 
                    [9, '2018Q1'], [3, '2018Q2'],
                    [5, '2018Q3'], [4, '2018Q4']],
                  columns=['Rate', 'Date'])
df1.index = pd.PeriodIndex(df1.Date, freq='Q')
df1.drop(columns='Date', inplace=True)

# This Code produces 'FutureWarning' message
df1['Update'] = np.where(np.in1d(df1.index - 3, df1.index.values),
                         df1.loc[df1.index - 3].Rate, df1.Rate.shift(1))

A complete warning message is as below:

FutureWarning: 
Passing list-likes to .loc or [] with any missing label will raise
KeyError in the future, you can use .reindex() as an alternative.

See the documentation here:
https://pandas.pydata.org/pandas-docs/stable/indexing.html#deprecate-loc-reindex-listlike

Link in warning: deprecate-loc-reindex-listlike

I understand that since there is arithmetic calculation on my index and some of the index value on calculation is not present which is causing this warning message and suggesting to use pandas.DataFrame.reindex. Although my intention is to access Rate only if index is available based on the True or False value from the condition in np.where but I guess numpy calculates for all and hence the missing index.

I tried by expanding the Date range, but it will have the same issue near the boundary. So, now my question is how to resolve this warning message. Any insights / thoughts / info / links would be helpful.

pandas python
1 Answers

The problem is that in preparation for what np.where may or may not need, df.loc[df.index - 3] gets evaluated for all values of df.index - 3. The warning message is a good one and suggests that you should use df.reindex(df.index - 3) instead.

df1['Update'] = np.where(np.in1d(df1.index - 3, df1.index.values),
                         df1.reindex(df1.index - 3).Rate, df1.Rate.shift(1))
user_23366540
Related