Splitting dataframe according to the size of each group

Viewed 67

How can I convert the following code into simple one line dynamically in Pandas? It is like always I have range of three numbers, and finding if the data exists in between that or not?

dataA = data.groupby('ID').apply(lambda x:  (len(x) > 3) & (len(x) < 6))
dataB = data.groupby('ID').apply(lambda x:  (len(x) > 6) & (len(x) < 9))
dataC = data.groupby('ID').apply(lambda x:  (len(x) > 9) & (len(x) < 12))
1 Answers

I'm assuming your goal is to split your dataframe into groups depending on the size of each group.

You can avoid creating extra variables by using a dictionary. You can also avoid defining boundaries manually for each slice by using a list. Finally, you can calculate the size of each group in one operation.

L = [3, 6, 9, 12]

sizes = data.groupby('ID')['SOME_COL'].transform('size')  # SOME_COL can be any series

data = {}
for key, (len1, len2) in zip('ABC', zip(L, L[1:])):
    data[key] = data.loc[sizes.between(len1, len2, inclusive=False)]

Alternatively, you can formulate the above as a dictionary comprehension:

data = {key: data.loc[sizes.between(len1, len2, inclusive=False)] \
        for key, (len1, len2) in zip('ABC', zip(L, L[1:]))}
Related