TypeError: cannot subtract DatetimeArray from ndarray

user_11086864 0
Viewed 13510

I have two columns in my dataframe - Start date and Finish date (It is a timestamp ). I would like to calculate runtime by finding the difference. When I run the code below, I get the error: TypeError: cannot subtract DatetimeArray from ndarray. I am also unable to split the finished date column into date and time so I can calculate runtime in another manner.

# calculate runtime 
pd.to_datetime(df.start_date)
pd.to_datetime(df.finish_date)

diff = pd.to_datetime(df.finish_date) - pd.to_datetime(df.start_date)

This is how the dataframe looks like: df

file_name                unique_id                                      start_date                         finish_date
        
file1         1716b94a8d8d83c8fffe4bdd14d536ae1ee9cba6bf17e6...   2020-09-03T16:18:38.929863799Z          2020-09-03T16:20:17.615093582Z
        
file2         87ff84ab119b798312230fceb3a8730fe74669a373650a...   2020-09-03T16:26:25.075167073Z          2020-09-04T00:04:39.702686798Z

How can I -

  1. either find the difference between finish & start date?
  2. split the columns into date & time.
python-datetime pandas
1 Answers

Assume you want to calculate the number of days between the dates, then this is one solution:

import datetime as dt
diff = (pd.to_datetime(df.finish_date) - pd.to_datetime(df.start_date)).dt.days

EDIT

Another alternative is

Start = pd.to_datetime(df.finish_date)
End  = pd.to_datetime(df.start_date)
End.subtract(Start)

Example: Here I choose to compute the difference between dates in a df and now.

metric_id device_id            timestamp  cpu_5min  vol_max
0          device_1  2020-12-04 05:15:00     116.0    734.0
1          device_1  2020-12-04 05:30:00     213.0    325.0
2          device_1  2020-12-04 05:35:00     427.0    668.0
3          device_2  2020-12-04 05:15:00     540.0      NaN
4          device_2  2020-12-04 05:30:00     127.0      NaN
5          device_2  2020-12-04 05:35:00     654.0      NaN

and

df['tDATE'] = pd.to_datetime(df['timestamp'])
df['DIFF1'] = (df['tDATE'] - dt.datetime.now()).dt.days  #method 11
df['DIFF2'] = df['tDATE'].subtract(dt.datetime.now()) #method2

which returns

metric_id device_id            timestamp  cpu_5min  vol_max  \
0          device_1  2020-12-04 05:15:00     116.0    734.0   
1          device_1  2020-12-04 05:30:00     213.0    325.0   
2          device_1  2020-12-04 05:35:00     427.0    668.0   
3          device_2  2020-12-04 05:15:00     540.0      NaN   
4          device_2  2020-12-04 05:30:00     127.0      NaN   
5          device_2  2020-12-04 05:35:00     654.0      NaN   

metric_id               tDATE  difd  DIFF1                     DIFF2  
0         2020-12-04 05:15:00   -14    -14 -14 days +22:13:26.627607  
1         2020-12-04 05:30:00   -14    -14 -14 days +22:28:26.627607  
2         2020-12-04 05:35:00   -14    -14 -14 days +22:33:26.627607  
3         2020-12-04 05:15:00   -14    -14 -14 days +22:13:26.627607  
4         2020-12-04 05:30:00   -14    -14 -14 days +22:28:26.627607  
5         2020-12-04 05:35:00   -14    -14 -14 days +22:33:26.627607

EDIT: Working with Timestamps

From your comments below it is now evident that the examples above need preparation since you are working with this Timepstamps. As a note, this is why giving enough information when asking the question is important (e.g. what kind of data you are dealing with). This is even moe important when it comes to dates as there are many formats. Here is an example with the date format you gave in the comments:

import datetime as dt

Date = '2020-09-03T16:18:38.929863799Z'
Date2  = '2020-10-03T16:18:38.929863799Z'

What you have here are Timestamps, so your first step is to convert them to datetime and then use to_pydate (It use to be called Timestamp.to_datetime() but is now deprecated.

Date = pd.to_datetime(Date)
Date2  = pd.to_datetime(Date2)

DATE_1 = Date.to_pydatetime()
DATE_2 = Date2.to_pydatetime()

after which you can compute the difference

DIFF = (pd.to_datetime(DATE_1) -pd.to_datetime(DATE_2))

which is Timedelta('-30 days +00:00:00')

user_53636860
Related