Pandas: Final order of Dataframe groupby when sort=False

user_18727795 0
Viewed 98

According to the Pandas documentation on https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.groupby.html

sort : bool, default True

Sort group keys. Get better performance by turning this off. Note this does not influence the order of observations within each group. Groupby preserves the order of rows within each group.

This doesn't really describe if, when sort=False, the order of groups will be random or retain the original order before groupby. Any insights into the Mars behavior here?

pandas-groupby pandas dataframe python
1 Answers

Here is an example of what it means:

Data:

import pandas as pd

df = pd.DataFrame(data={"cost": [30, 29, 40, 45, 240, 200, 220, 180, 150, 130],
                        "date": [ "2017-11-01", "2016-12-01", "2016-11-01",
                                 "2017-12-01", "2018-11-30", "2018-12-01",
                                 "2019-11-30", "2019-12-30", "2020-11-30",
                                 "2020-12-30"]})

df["date"] = pd.to_datetime(df["date"])
df.set_index("date", inplace=True)

print(df)
#            cost
#date            
#2017-11-01    30
#2016-12-01    29
#2016-11-01    40
#2017-12-01    45
#2018-11-30   240
#2018-12-01   200
#2019-11-30   220
#2019-12-30   180
#2020-11-30   150
#2020-12-30   130

Notice that the first date is 2017.

With sort=False 2017 is still the first row:

df.groupby([lambda x: x.year], sort=False).count()
#Out: 
#      cost
#date      
#2017     2
#2016     2
#2018     2
#2019     2
#2020     2

As the default is sort=True, 2017 is no longer first, as the dates are sorted:

df.groupby([lambda x: x.year]).count()
#Out: 
#      cost
#date      
#2016     2
#2017     2
#2018     2
#2019     2
#2020     2

Without sorting (sort=False), the groups are in the order of first appearance in the row. With sorting (sort=True or not given), the groups will be sorted as Python sorts, i.e. numerically, alphabetically etc.

user_185715650
Related