Having an issue converting a set of very large integers into timedeltas

Viewed 28

I am working with a dataframe that includes a column of integers where the units are the number of days since 0001-01-01. I need to convert these integers into current dates. When I attempt use the pd.to_timedelta function to convert these integers into TimeDeltas that I can then add to the start date, the resulting time deltas do not result in what I expect. Please run the following code for an example:

import pandas as pd
df = pd.DataFrame([735110,735111,735112,735114], columns=["days_since_0001-01-01"])
df['time_added'] = pd.to_timedelta(df['days_since_0001-01-01'], unit='d')
print(df[0:1])
#output: 735110 94598 days 01:16:18.871345152

As you can see, the timedelta result for the first row in days is 94,598, plus additional units for the hours/seconds/minutes etc. What I was expecting for that row was 735,110 days, plus 0s for the rest of the timestamp. Further, if I use a DataFrame with a smaller number of days, the output is as I would expect. I have come to the conclusion that to_timedelta cannot handle very large numbers of days; however, I do not know of an alternative method to do this. I could simply reduce the number of days by an arbitrary amount and increase the start date, but I would still need to know the proper amount by which to reduce the days integer and the amount by which to increase the start date. Any help would be appreciated.

1 Answers

Every year is 365 days, except for leap years which are 366 days and occur every four years. So I just need to pick an arbitrary point within the acceptable timedelta resolutions, (e.x. 1850), calculate the number of regular years and leap years between 0000-01-01 and 1850-01-01, multiply regular by 365 and leap by 366, then subtract that from my days_since_0001-01-01. Then I can set a start date as 1850-01-01, and add the remaining amount of days to that start date, to get the current date.

Related