I have a df like below
col1, mydate
1, 25-DEC-2016 09:15:00
2, 25-DEC-2016 10:14:00
3, 25-DEC-2016 10:16:00
4, 25-DEC-2016 10:18:56
2, 25-DEC-2016 11:14:00
2, 25-DEC-2016 10:16:00
df.info(): mydate 323809 non-null object
I need to this dataframe according to time, like df having time less than 10:15:00, df having time less than 11:15:00
So created my slice intervals using
times=[pd.to_datetime(i) for i in '10:15:00','11:15:00','12:15:00','13:15:00','14:15:00','15:15:00', '15:30:00']
Then I convert my mydate type to time which takes a lot of time
df['mydate']=df4.mydate.apply(lambda x: pd.to_datetime(x,infer_datetime_format=True).time())
The above command I think can be optimised, or there should be a better/faster way.
Then I simply do
for time in times:
slice = df[df.mydate<time.time()]
My intent is only to compare df.mydate time with ['10:15:00','11:15:00','12:15:00','13:15:00','14:15:00','15:15:00', '15:30:00'] (but not dates) and simply subset df
The above way works fine for me but I am looking for a better way.
Additional: Interestingly sorting mydate was very fast (even though I did not convert to mydate col to datetime) using
df.sort_values(by='mydate')
which lets me think that my way of subsetting should be faster.
mydate col will always be in 25-DEC-2016 09:15:00 format (Note DEC not Dec) can I use format='%d-%b-%Y %H:%M:%S'