grouping data using pandas datetime inot 12 hour shifts 7am - 7pm and 7pm - 7am the next day

user_14086606 0
Viewed 180

I have a data pandas dataframe that has two columns. It has a column called 'DateAndTime' (datetime64[ns]) and a column called 'Finished' (Bool). There are some 5000+ rows, all with different date and time, and with the 'Finished' column as True.

What I am trying to do is to group the data into 'shifts' of 7am-7pm and 7pm-7am and sum how many Trues occurred in the 12hr period.

df.head()

DateAndTime                   Finished
109 2020-07-28 14:36:07.983     True
110 2020-07-28 14:36:34.547     True
111 2020-07-28 14:39:38.187     True
112 2020-07-28 14:41:10.547     True
113 2020-07-28 14:41:32.250     True

df.describe()

       DateAndTime                    Finished
count   5915                            5915
unique  5915                            2
top     2020-07-29 07:34:25.360000      True
freq    1                               5914
first   2020-07-28 14:36:07.983000      NaN
last    2020-08-05 04:57:10.657000      NaN
pandas datetime python
2 Answers

You should try this

import numpy as np

#in case columns are in String Format 
df = df.astype({'DateAndTime': np.datetime64, 'Finished':np.bool}) 

# 7AM : 7PM Shift
shift_1 = df[df.DateAndTime.apply(lambda t: (t.hour in range(7, 19)) or (t.hour==19 and (t.second+t.minute==0)))]

# 7PM : 7AM Shift
shift_2 = df[df.DateAndTime.apply(lambda t: not ((t.hour in range(7, 19)) or (t.hour==19 and (t.second+t.minute==0))))]

shift_1_TruedCount = shift_1.Finished.to_list().count(True)
shift_2_TruedCount = shift_2.Finished.to_list().count(True)

print(shift_1_TruedCount, shift_2_TruedCount)
user_124965090

Assuming your DateAndTime column is already of type Timestamp:

# Move DateAndTime back by 7 hours
# Now shift 1 is 0:00 to 12:00, shift 2 is 12:00 - 24:00
h = (df['DateAndTime'] - pd.Timedelta(hours=7)).dt.hour < 12
df['Shift'] = h.map({True: '7am-7pm', False: '7pm-7am'})
user_25389390
Related