How to create the datetime object equivalent of numpy.nan without using Pandas?

Viewed 4041

Given a numpy array of numerical data, one can retrieve the indices of the array that match a particular condition. By accepting np.nan as type <float>, these conditions can still be met. I would like to use a similar method using datetime objects instead of numerical data.

As an example, consider the following:

import numpy as np
import datetime

# sample data
x = np.linspace(-5, 5, 11).astype(int)
y = x**2
z = y.copy().astype(float)
z[0] = np.nan

# check that nan condition is correctly satisfied
print("\nZ:\n{}\n".format(z))
print("\nZ > 0:\n{}\n".format(z > 0))
print("\nZ < 0:\n{}\n".format(z < 0))
print("\nZ = 0:\n{}\n".format(z == 0))

# get number of elapsed hours between two datetime objects
dt0 = datetime.datetime(2018, 1, 11, 23, 59, 59)
dt1 = datetime.datetime(2018, 1, 12, 23, 59, 59)
hr_delta = (dt1 - dt0).total_seconds() // 3600
print("\n{} - {} = {} hours".format(dt1, dt0, hr_delta))

Since np.nan - 5 = np.nan, it would be very convenient for me to use this idea to create an invalid datetime object. However, datetime objects require the inputs of type <int>, whereas np.nan is type <float>. The example below recreates the source of my problem:

# attempt to instantiate invalid datetime object
dt2 = datetime.datetime(np.nan, 1, 11, 23, 59, 59)

which throws TypeError: integer argument expected, got float. Is there a way to do this without using pandas, as suggested in this alternate post? Just to clarify, my goal is to create an invalid datetime object such that its subtraction or addition with a valid datetime object returns a nan-like value.

1 Answers

Anything pandas does is usually something Numpy does.

In this instance, you need to store the proper type, which is not datetime, but np.datetime64. Once you have this type, you can fill NaT values with np.datetime64("NaT") and then use np.isnat to test if a value if a time or not.

datetime itself doesn't provide a concept of "not a time", you have to use a type that does. You should be able to use np.datetime64 in a close fashion as a datetime.

Related