Using pandas groupby to find the average length of the text within each group

user_7148245 0
Viewed 7889

I'm using the shakespeare corpus. 

    act literature_type scene   scene_text  scene_title speaker title
0   1   Comedy  1   In delivering my son from me, I bury a second ...   Rousillon. The COUNT's palace.  COUNTESS    All's Well That Ends Well
1   1   Comedy  1   And I in going, madam, weep o'er my father's d...   Rousillon. The COUNT's palace.  BERTRAM All's Well That Ends Well
2   1   Comedy  1   You shall find of the king a husband, madam; y...   Rousillon. The COUNT's palace.  LAFEU   All's Well That Ends Well
3   1   Comedy  1   What hope is there of his majesty's amendment?  Rousillon. The COUNT's palace.  COUNTESS    All's Well That Ends Well
4   1   Comedy  1   He hath abandoned his physicians, madam; under...   Rousillon. The COUNT's palace.  LAFEU   All's Well That Ends Well

I'd like to find the average scene_text length for each title.

I thought to use something along the lines of:

all_works_by_speaker_df.groupby('title').apply(lambda x: np.mean(len(x)))

This just returns the number of scenes in each title.

pandas-groupby pandas python
3 Answers

If need lens of characters:

df = (all_works_by_speaker_df.groupby('title')['scene_text']
                            .apply(lambda x: np.mean(x.str.len()))
                            .reset_index(name='mean_len_text'))
print (df)

                       title  mean_len_text
0  All's Well That Ends Well           48.4

If need lens of words use Vaishali's solution.

user_29010020

Take the lengths of the strings from the column, then groupby an array which is your play title, then apply the mean.

mean_len = df.scene_text.str.len().groupby(df.title).mean()
user_12527590

Split, len and mean

df.groupby('title').scene_text.apply(lambda x: x.str.split().str.len().mean())


title
All's Well That Ends Well    9.2
user_62873080
Related