Create pandas DatetimeIndex without leap day

Question

I want to create a pandas DatetimeIndex that contains a range of dates around the leap day for various years. Some years are a leap year and some are not. The caveat here though is that I want all of these lists of dates to be the same length. Let's look at some examples.

import pandas as pd
from datetime import timedelta

leap=pd.date_range('2020-02-27-12',pd.to_datetime('2020-02-27-12')+dt.timedelta(days=2),freq='6H')

DatetimeIndex(['2020-02-27 12:00:00', '2020-02-27 18:00:00',
               '2020-02-28 00:00:00', '2020-02-28 06:00:00',
               '2020-02-28 12:00:00', '2020-02-28 18:00:00',
               '2020-02-29 00:00:00', '2020-02-29 06:00:00',
               '2020-02-29 12:00:00'],
              dtype='datetime64[ns]', freq='6H')
len(leap)
9

One of the most common approaches to avoiding leap days in this format is to simply remove them from the list.

leap=leap[(leap.day != 29) | (leap.month != 2)]
len(leap)
6

This works fine if I just want to remove the leap day and don't care about the specific length of the list.

Let's do the same exercise for a non-leap year.

leap=pd.date_range('2021-02-27-12',pd.to_datetime('2021-02-27-12')+dt.timedelta(days=2),freq='6H')

DatetimeIndex(['2021-02-27 12:00:00', '2021-02-27 18:00:00',
               '2021-02-28 00:00:00', '2021-02-28 06:00:00',
               '2021-02-28 12:00:00', '2021-02-28 18:00:00',
               '2021-03-01 00:00:00', '2021-03-01 06:00:00',
               '2021-03-01 12:00:00'],
              dtype='datetime64[ns]', freq='6H')

len(leap)
9

Naturally, the first two have the same length. However, when we remove the leap dates from the first list we can now see that the lengths are different, as 6 =/ 9.

This leads to the ultimate question: how can I create a pandas date range that would take the first list and skip over the leap days and go straight to March, keeping the length of the list at 9?

For context, I am indexing geospatial (3D) data using a centered date approach. For example, if I want to look at data at some point on the globe on Jan 5, I analyze the data 2 days before and after Jan 5 (Jan 3-7). In doing this approach over a long period of time (>30 years) I get a better climatological sense of the variable I am analyzing. To do this indexing I am looping over dates and using the pandas date range approach described above. This is the loop I am using to approach a problem in indexing geospatial (3D) data:

times=pd.date_range('1979-09-01','1980-04-30-18', freq='6H')
final_times = times[(times.day != 29) | (times.month != 2)]
years=np.arange(1979,2020,1)
for i in final_times:
    print(i)

    times_list=[]

    for j in years:
        times_forward=pd.date_range(i.replace(year=j),i.replace(year=j)+dt.timedelta(days=2), freq='6H')
        times_back=pd.date_range(i.replace(year=j)-dt.timedelta(days=2),i.replace(year=j)-dt.timedelta(hours=6), freq='6H')
        total_times=times_forward.union(times_back)
        times_list.append(total_times)
    combined_times=pd.DatetimeIndex([item for sublist in times_list for item in sublist]).sort_values()

When running this code as is the date lists that surround the leap day are shorter in length than those that are not near the leap day.

user_1387346 · Answer

If you just use DateOffset objects, the problem is sort of handled for you:

>>> pd.date_range('2021-02-27-12', periods=9, freq='6H')
DatetimeIndex(['2021-02-27 12:00:00', '2021-02-27 18:00:00',
               '2021-02-28 00:00:00', '2021-02-28 06:00:00',
               '2021-02-28 12:00:00', '2021-02-28 18:00:00',
               '2021-03-01 00:00:00', '2021-03-01 06:00:00',
               '2021-03-01 12:00:00'],
              dtype='datetime64[ns]', freq='6H')
>>> pd.date_range('2021-02-27-12', periods=9, freq='6H') - pd.DateOffset(years=1)
DatetimeIndex(['2020-02-27 12:00:00', '2020-02-27 18:00:00',
               '2020-02-28 00:00:00', '2020-02-28 06:00:00',
               '2020-02-28 12:00:00', '2020-02-28 18:00:00',
               '2020-03-01 00:00:00', '2020-03-01 06:00:00',
               '2020-03-01 12:00:00'],
              dtype='datetime64[ns]', freq=None)

This does what you want, as long as you reference year is not a leap year. Note that if you’re basing yourself off a leap year you’ll have twice the same date, i.e. you’ll compare both the 28th and 29th to the 28th of the previous or next year:

>>> pd.date_range('2020-02-27-12', periods=9, freq='6H')
DatetimeIndex(['2020-02-27 12:00:00', '2020-02-27 18:00:00',
               '2020-02-28 00:00:00', '2020-02-28 06:00:00',
               '2020-02-28 12:00:00', '2020-02-28 18:00:00',
               '2020-02-29 00:00:00', '2020-02-29 06:00:00',
               '2020-02-29 12:00:00'],
              dtype='datetime64[ns]', freq='6H')
>>> pd.date_range('2020-02-27-12', periods=9, freq='6H') + pd.DateOffset(years=1)
DatetimeIndex(['2021-02-27 12:00:00', '2021-02-27 18:00:00',
               '2021-02-28 00:00:00', '2021-02-28 06:00:00',
               '2021-02-28 12:00:00', '2021-02-28 18:00:00',
               '2021-02-28 00:00:00', '2021-02-28 06:00:00',
               '2021-02-28 12:00:00'],
              dtype='datetime64[ns]', freq=None)

I’m not sure if this is problematic − the index then contains duplicated items but that’s about the only drawback I suppose.

Based on this, here’s a function that gives, for a given date, the index of each year’s dates (in the range) to compare:

def compare_times(datetime, periods=9, freq='6H', years=np.arange(1979, 2021)):
    ref_index = pd.DatetimeIndex([
        *pd.date_range(datetime, periods=1 + periods // 2, freq=f'-{freq}')[::-1],
        *pd.date_range(datetime, periods=1 + periods // 2, freq=freq)[1:]
    ])
    return pd.DatetimeIndex(np.concatenate([ref_index + pd.DateOffset(years=y -datetime.year) for y in years]))

There is now no issue anymore, as long as you don’t call the function with a 29th of February. And if you do, non-leap years will return the same number of items but with duplicate entries for the 28th of February:

>>> compare_times(pd.Timestamp(2020, 2, 29))
DatetimeIndex(['1979-02-28 00:00:00', '1979-02-28 06:00:00',
               '1979-02-28 12:00:00', '1979-02-28 18:00:00',
               '1979-02-28 00:00:00', '1979-02-28 06:00:00',
               '1979-02-28 12:00:00', '1979-02-28 18:00:00',
               '1979-03-01 00:00:00', '1980-02-28 00:00:00',
               ...
               '2019-03-01 00:00:00', '2020-02-28 00:00:00',
               '2020-02-28 06:00:00', '2020-02-28 12:00:00',
               '2020-02-28 18:00:00', '2020-02-29 00:00:00',
               '2020-02-29 06:00:00', '2020-02-29 12:00:00',
               '2020-02-29 18:00:00', '2020-03-01 00:00:00'],
              dtype='datetime64[ns]', length=378, freq=None)

Here you can see the duplicates in 1979.

Create pandas DatetimeIndex without leap day

1 Answers