Pandas understanding groupby

user_2543622 0
Viewed 37

I have a dataframe as below

import pandas as pd
import numpy as np
df = {'a': ['xxx', 'xxx','xxx','yyy','yyy','yyy'], 'start': [10000, 10500, 11000, 12000, 13000, 14000], 'day':['2018-10-08',
'2018-09-26',
'2018-08-07',
'2018-10-04',
'2019-02-06',
'2019-02-10'] }
df = pd.DataFrame(data=df)
df["day2"] = pd.to_datetime(df['day']).dt.date
print (df)
print (df.dtypes)



     a  start         day        day2
0  xxx  10000  2018-10-08  2018-10-08
1  xxx  10500  2018-09-26  2018-09-26
2  xxx  11000  2018-08-07  2018-08-07
3  yyy  12000  2018-10-04  2018-10-04
4  yyy  13000  2019-02-06  2019-02-06
5  yyy  14000  2019-02-10  2019-02-10
a        object
start     int64
day      object
day2     object
dtype: object

When I used as_index=False, I get output as below - it makes sense

df.groupby('a',as_index=False).agg({'start':'first','day2':'min'})
    a   start   day2
0   xxx 10000   2018-08-07
1   yyy 12000   2018-10-04

question 1) But why dont I get the column a when I run below?

df2=df.copy()
df4=df2.groupby('a',as_index=False)['day2'].apply(list)
print(df4)

0    [2018-10-08, 2018-09-26, 2018-08-07]
1    [2018-10-04, 2019-02-06, 2019-02-10]
dtype: object
===============================

I get that column if I don't use as_index=False.

print ("===============================")
print (df2.groupby('a')['day2'].apply(list))
a
xxx    [2018-10-08, 2018-09-26, 2018-08-07]
yyy    [2018-10-04, 2019-02-06, 2019-02-10]










question 2) why `transform` is not working?

df2.groupby('a')['day2'].transform(list)
0    2018-10-08
1    2018-09-26
2    2018-08-07
3    2018-10-04
4    2019-02-06
5    2019-02-10
Name: day2, dtype: object

question 3) is there any list of functions such as max, min, first, last that I could use along with groupby?

pandas-groupby pandas group-by python
1 Answers

Your 1st question with apply it will not show the column , but we can do agg

df2.groupby('a',as_index=False)['day2'].agg(list)
Out[15]: 
     a                                  day2
0  xxx  [2018-10-08, 2018-09-26, 2018-08-07]
1  yyy  [2018-10-04, 2019-02-06, 2019-02-10]

To fix the transform

df2.groupby('a')['day2'].transform(lambda x : [x.tolist()]*len(x))
Out[19]: 
0    [2018-10-08, 2018-09-26, 2018-08-07]
1    [2018-10-08, 2018-09-26, 2018-08-07]
2    [2018-10-08, 2018-09-26, 2018-08-07]
3    [2018-10-04, 2019-02-06, 2019-02-10]
4    [2018-10-04, 2019-02-06, 2019-02-10]
5    [2018-10-04, 2019-02-06, 2019-02-10]
Name: day2, dtype: object

However, for you 2nd question below is better

df2.a.map(df2.groupby('a')['day2'].agg(list))

For your 3rd question

df2.groupby(a)['day2'].agg(['max', 'min', 'first', 'last'])
user_79645270
Related