When running groupby-agg on a Dask dataframe, the resulting Dask dataframe is not indexed. If the groupby is run on a single column, is it possible to achieve an indexed Dask dataframe?
from dask.datasets import timeseries
df = timeseries()
a = df.groupby('name').agg('sum')
print(a.known_divisions) # False
To get the indexed dataframe, it's possible to do:
b = a.reset_index().set_index('name')
print(b.known_divisions) # True
However, the .set_index operation will shuffle data, which might be possible to avoid at the time of groupby-agg.
Is there any other set of operations that will eventually give the aggregated dataframe such that a.known_divisions==True?
Update: the specific use case I have in mind is when the unique values of name column are known (and there could be many, many unique values). For example, pretend there is a million names, then it would be great to get the groupby-agg result, so that all names starting with 'A' are in one partition, with 'B' in second partition, etc.