Join rows in the same data frame

user_1112325 0
Viewed 54

I have a data frame that I want to make some changes. Here's a sample:

d = {'username': ['a', 'a', 'b', 'a', 'a'],
     'state': ['AR', 'AZ', 'CA', 'CO', 'NY'],
     'status': ['ADD', 'ADD', 'REMOVE', 'ADD', 'REMOVE']}
df = pd.DataFrame(data=d)

I know how I can groupby and join the states:

df = df.fillna('').groupby(['username', 'status'], as_index=False)['state'] \
    .apply(lambda x: ','.join(set(x))) \
    .reset_index() \
    .rename({0: 'state'}, axis=1)

But at the end I have something like this but still not what I need:

username  status     state
a         ADD        AR,AZ,CO
a         REMOVE     NY
b         REMOVE     CA

I want to produce this final report:

username  ADD      REMOVE
a         AR,AZ,CO NY   
b                  CA

Any ideas?

Thank you very much!

pandas-groupby pandas python-3.x python
2 Answers

You are close, before reset_index use Series.unstack:

df1 = (df.fillna('')
         .groupby(['username', 'status'])['state'] \
         .apply(lambda x: ','.join(set(x)))
         .unstack(fill_value='')
         .reset_index()
         .rename_axis(None, axis=1))
print (df1)
  username       ADD REMOVE
0        a  AZ,AR,CO     NY
1        b               CA

Or use DataFrame.pivot_table with convert index to column by reset_index and removing column name by DataFrame.rename_axis:

df1 = (df.pivot_table(index='username', 
                     columns='status', 
                     values='state', 
                     aggfunc=lambda x: ','.join(set(x)), 
                     fill_value='')
         .reset_index()
         .rename_axis(None, axis=1))

print (df1)
  username       ADD REMOVE
0        a  AZ,AR,CO     NY
1        b               CA

EDIT:

Solution with removing sets is possible if use DataFrame.drop_duplicates by 3 columns:

Changed sample data for better explanation:

d = {'username': ['a', 'a', 'b', 'a', 'a', 'a'],
     'state': ['AR', 'AZ', 'CA', 'CO', 'NY', 'NY'],
     'status': ['ADD', 'ADD', 'REMOVE', 'ADD', 'REMOVE','REMOVE']}
df = pd.DataFrame(data=d)
print (df)
  username state  status
0        a    AR     ADD
1        a    AZ     ADD
2        b    CA  REMOVE
3        a    CO     ADD
4        a    NY  REMOVE
5        a    NY  REMOVE <- added row

df1 = (df.pivot_table(index='username', 
                     columns='status', 
                     values='state', 
                     aggfunc=lambda x: ','.join(set(x)), 
                     fill_value='')
         .reset_index()
         .rename_axis(None, axis=1))

print (df1)
  username       ADD REMOVE
0        a  AZ,AR,CO     NY
1        b               CA

df1 = (df.drop_duplicates(['username','status','state'])
         .pivot_table(index='username', 
                     columns='status', 
                     values='state', 
                     aggfunc= ','.join, 
                     fill_value='')
         .reset_index()
         .rename_axis(None, axis=1))

print (df1)
  username       ADD REMOVE
0        a  AZ,AR,CO     NY
1        b               CA
user_29010020

We can use pivot_table with custom aggfunc here:

piv = df.pivot_table(index='username', columns='status', values='state', aggfunc=','.join)

status         ADD REMOVE
username                 
a         AR,AZ,CO     NY
b              NaN     CA
user_90812670
Related