df.groupby(.pdGrouper(freq='D') ).idxmax throwing an ValueError

Viewed 160

I'm attempting to groupby the day and locate the max value for each day.

df.loc[df.groupby(pd.Grouper(freq='D'))['High'].idxmax()]

When my dataframe is small with only a few consecutive days it successfully runs (hits an error with weekends/holidays missing I think)

                          InternalSymbol    Open    High     Low   Close  
CollectionTimestamp (CST)                                                  
2018-09-17 10:00:00                 GCZ8  1206.1  1209.7  1205.9  1208.3   
2018-09-18 09:00:00                 GCZ8  1205.6  1208.2  1205.2  1206.4   
2018-09-19 08:00:00                 GCZ8  1207.6  1211.0  1205.9  1210.0   
2018-09-20 10:00:00                 GCZ8  1211.0  1213.2  1208.1  1209.1   
2018-09-21 06:00:00                 GCZ8  1212.3  1212.7  1209.3  1209.7 

When I try to incorrporate my entire dataset (a few years, Sun-Fri, excluding market holidays etc). I receive this error..

ValueError: attempt to get argmax of an empty sequence

I'm assuming it has something to do with my line of code in regards to the freq='D'. Is there a work around for something like this?

Here is the example dataframe. Note I left in 2018-09-24 data so it will hit the error I'm refering to, if you delete the 2018-09-24 lines, then the .pdGrouper(freq='D') will work.

                          InternalSymbol    Open    High     Low   Close
CollectionTimestamp (CST)                                               
2018-09-17 06:00:00                 GCZ8  1202.1  1203.7  1201.5  1202.5
2018-09-17 07:00:00                 GCZ8  1202.5  1202.6  1200.8  1202.2
2018-09-17 08:00:00                 GCZ8  1202.3  1204.8  1202.1  1204.1
2018-09-17 09:00:00                 GCZ8  1204.1  1206.7  1203.4  1206.1
2018-09-17 10:00:00                 GCZ8  1206.1  1209.7  1205.9  1208.3
2018-09-17 11:00:00                 GCZ8  1208.2  1209.0  1207.0  1207.8
2018-09-17 12:00:00                 GCZ8  1207.8  1207.9  1206.5  1207.3
2018-09-17 13:00:00                 GCZ8  1207.2  1207.4  1205.3  1205.9
2018-09-17 14:00:00                 GCZ8  1205.9  1206.8  1204.6  1205.6
2018-09-17 15:00:00                 GCZ8  1205.7  1206.0  1204.6  1205.2
2018-09-18 06:00:00                 GCZ8  1203.7  1204.9  1202.9  1204.7
2018-09-18 07:00:00                 GCZ8  1204.8  1207.8  1204.6  1207.0
2018-09-18 08:00:00                 GCZ8  1207.0  1207.1  1204.1  1205.7
2018-09-18 09:00:00                 GCZ8  1205.6  1208.2  1205.2  1206.4
2018-09-18 10:00:00                 GCZ8  1206.3  1206.5  1202.2  1204.6
2018-09-18 11:00:00                 GCZ8  1204.7  1205.0  1203.1  1203.8
2018-09-18 12:00:00                 GCZ8  1203.8  1204.3  1202.7  1203.0
2018-09-18 13:00:00                 GCZ8  1203.0  1203.9  1201.9  1203.0
2018-09-18 14:00:00                 GCZ8  1203.0  1203.3  1201.7  1202.4
2018-09-18 15:00:00                 GCZ8  1202.3  1203.0  1201.4  1202.9
2018-09-19 06:00:00                 GCZ8  1207.3  1208.4  1207.2  1207.6
2018-09-19 07:00:00                 GCZ8  1207.6  1208.2  1206.8  1207.6
2018-09-19 08:00:00                 GCZ8  1207.6  1211.0  1205.9  1210.0
2018-09-19 09:00:00                 GCZ8  1210.0  1210.1  1206.0  1207.9
2018-09-19 10:00:00                 GCZ8  1207.9  1208.9  1205.7  1208.1
2018-09-19 11:00:00                 GCZ8  1208.1  1210.8  1207.4  1208.2
2018-09-19 12:00:00                 GCZ8  1208.3  1209.5  1208.1  1208.9
2018-09-19 13:00:00                 GCZ8  1208.9  1209.0  1207.2  1207.5
2018-09-19 14:00:00                 GCZ8  1207.4  1208.9  1207.4  1208.6
2018-09-19 15:00:00                 GCZ8  1208.6  1208.7  1207.6  1208.3
2018-09-20 06:00:00                 GCZ8  1207.4  1208.3  1206.8  1207.8
2018-09-20 07:00:00                 GCZ8  1207.8  1210.4  1207.2  1210.2
2018-09-20 08:00:00                 GCZ8  1210.2  1212.5  1209.7  1211.7
2018-09-20 09:00:00                 GCZ8  1211.8  1212.4  1209.8  1211.0
2018-09-20 10:00:00                 GCZ8  1211.0  1213.2  1208.1  1209.1
2018-09-20 11:00:00                 GCZ8  1209.1  1209.6  1207.6  1208.2
2018-09-20 12:00:00                 GCZ8  1208.2  1210.5  1208.0  1210.4
2018-09-20 13:00:00                 GCZ8  1210.3  1211.6  1209.5  1210.6
2018-09-20 14:00:00                 GCZ8  1210.5  1211.4  1209.6  1211.3
2018-09-20 15:00:00                 GCZ8  1211.4  1212.6  1211.2  1211.9
2018-09-21 06:00:00                 GCZ8  1212.3  1212.7  1209.3  1209.7
2018-09-21 07:00:00                 GCZ8  1209.7  1210.4  1208.6  1209.1
2018-09-21 08:00:00                 GCZ8  1209.0  1209.8  1199.3  1200.1
2018-09-21 09:00:00                 GCZ8  1200.2  1202.0  1196.0  1200.8
2018-09-21 10:00:00                 GCZ8  1200.9  1205.6  1200.9  1204.3
2018-09-21 11:00:00                 GCZ8  1204.3  1204.7  1199.8  1200.6
2018-09-21 12:00:00                 GCZ8  1200.6  1203.0  1200.3  1201.7
2018-09-21 13:00:00                 GCZ8  1201.6  1203.3  1200.3  1201.3
2018-09-21 14:00:00                 GCZ8  1201.4  1201.5  1200.2  1201.5
2018-09-21 15:00:00                 GCZ8  1201.6  1203.5  1201.5  1203.3
2018-09-24 06:00:00                 GCZ8  1203.3  1204.1  1202.9  1203.0
2018-09-24 07:00:00                 GCZ8  1203.0  1204.9  1202.9  1204.2
2018-09-24 08:00:00                 GCZ8  1204.3  1205.4  1202.5  1202.6
2018-09-24 09:00:00                 GCZ8  1202.5  1208.6  1202.3  1207.1
2018-09-24 10:00:00                 GCZ8  1207.1  1208.8  1205.9  1207.8
2018-09-24 11:00:00                 GCZ8  1207.7  1208.3  1204.7  1206.1
2018-09-24 12:00:00                 GCZ8  1206.1  1206.2  1204.7  1205.6
2018-09-24 13:00:00                 GCZ8  1205.5  1205.5  1203.5  1204.2
2018-09-24 14:00:00                 GCZ8  1204.1  1204.3  1203.3  1203.8
2018-09-24 15:00:00                 GCZ8  1203.7  1204.2  1202.9  1203.3
1 Answers

Try grouping on the existing days. Using grouper or resample will attempt to fill in days you're missing with NaNs which don't have a maximum so to speak so there's no existing index that associates with those missing days:

df.loc[df.groupby(df.index.date)["High"].idxmax()]


                          InternalSymbol    Open    High     Low   Close
CollectionTimestamp (CST)                                               
2018-09-17 10:00:00                 GCZ8  1206.1  1209.7  1205.9  1208.3
2018-09-18 09:00:00                 GCZ8  1205.6  1208.2  1205.2  1206.4
2018-09-19 08:00:00                 GCZ8  1207.6  1211.0  1205.9  1210.0
2018-09-20 10:00:00                 GCZ8  1211.0  1213.2  1208.1  1209.1
2018-09-21 06:00:00                 GCZ8  1212.3  1212.7  1209.3  1209.7
2018-09-24 10:00:00                 GCZ8  1207.1  1208.8  1205.9  1207.8
Related