Different position of subscript operator leads to the same result in pandas groupby

user_9170979 0
Viewed 92

I am learning pandas through a course. In the groupby video of the course I got confused over this statement.

df.set_index('STNAME').groupby(level=0)['CENSUS2010POP'].agg(avg=np.average, sum=np.sum).head()

Output is:

The output

In this statement the subscript operator ['CENSUS2010POP'] is placed after groupby. My question is that groupby returns a groupby object in which it has tuples. The tuples are pairs of the data on which grouping was done and dataframes based on that grouping condition. How can we subscript a groupby object in the first place. Moreover when I write the query in the following way it still works.

df.set_index('STNAME')['CENSUS2010POP'].groupby(level=0).agg(avg=np.average, sum=np.sum).head()

Why does placing the subscript operator after groupby work? Why do both statements work the same way? Is there even a difference between both statements?

pandas-groupby data-science pandas dataframe python
2 Answers

They are different:

  • df.set_index('STNAME').groupby(level=0)['CENSUS2010POP'].agg(avg=np.average, sum=np.sum).head()
  • df.set_index('STNAME')['CENSUS2010POP'].groupby(level=0).agg(avg=np.average, sum=np.sum).head()

In the first case you group all columns, take CENSUS2010POP, and average and sum, while the second case you take CENSUS2010POP, group, and take average and sum.

user_89806340

Here is a way to see intermediate results in the .groupby() calculation. First, create a simple data set:

import pandas as pd
data = {'Age': {0: 10, 1: 11, 2: 12, 3: 13},
 'Animal': {0: 'Falcon', 1: 'Falcon', 2: 'Parrot', 3: 'Parrot'},
 'Max Speed': {0: 390.0, 1: 350.0, 2: 30.0, 3: 20.0},
 'Type': {0: 'Captive', 1: 'Wild', 2: 'Captive', 3: 'Wild'}}
df = pd.DataFrame(data)
print(pd.DataFrame(data))

   Age  Animal  Max Speed     Type
0   10  Falcon      390.0  Captive
1   11  Falcon      350.0     Wild
2   12  Parrot       30.0  Captive
3   13  Parrot       20.0     Wild

Second, show a typical calculation:

df.groupby('Animal')['Max Speed'].mean()

Animal
Falcon    370.0
Parrot     25.0
Name: Max Speed, dtype: float64

Third, show intermediate results:

print('0 ', df.groupby('Animal').groups                                 , end='\n\n')
print('1 ', df.groupby('Animal')                                        , end='\n\n')
print('2 ', df.groupby('Animal').get_group('Falcon')                    , end='\n\n')
print('3 ', df.groupby('Animal').get_group('Falcon')['Max Speed']       , end='\n\n')
print('4 ', df.groupby('Animal').get_group('Falcon')['Max Speed'].mean(), end='\n\n')

0  {'Falcon': Int64Index([0, 1], dtype='int64'), 
    'Parrot': Int64Index([2, 3], dtype='int64')}

1  <pandas.core.groupby.generic.DataFrameGroupBy object at 0x7fc5499a9a00>

2     Age  Animal  Max Speed     Type
0   10  Falcon      390.0  Captive
1   11  Falcon      350.0     Wild

3  0    390.0
1    350.0
Name: Max Speed, dtype: float64

4  370.0
user_136085990
Related