Daily Mentions of a Word

user_8618380 0
Viewed 94

I have the following df, containing daily articles from different sources:

print(df)

Date         content

2018-11-01    Apple Inc. AAPL 1.54% reported its fourth cons...
2018-11-01    U.S. stocks climbed Thursday, Apple is a real ...
2018-11-02    GONE are the days when smartphone manufacturer...
2018-11-03    To historians of technology, the story of the ...
2018-11-03    Apple Inc. AAPL 1.54% reported its fourth cons...
2018-11-03    Apple is turning to traditional broadcasting t...

(...)

I would like to compute the total number of daily mentions - hence aggregating by Date - of the word "Apple". How can I create "final_df"?

print(final_df) 

    2018-11-01    2
    2018-11-02    0
    2018-11-03    2
    (...)
pandas word-count nlp python
3 Answers

Use count for new Series, aggregate by column df['Date'] with sum:

df1 = df['content'].str.count('Apple').groupby(df['Date']).sum().reset_index(name='count')
print (df1)
         Date  count
0  2018-11-01      2
1  2018-11-02      0
2  2018-11-03      2
user_29010020

You can GroupBy the different dates, use str.count to count the occurrences of Apple and aggregate with the sum to get the amount of counts in each group:

df.groupby('Date').apply(lambda x: x.content.str.count('Apple').sum())
                  .reset_index(name='counts')

      Date     counts
0 2018-11-01       2
1 2018-11-02       0
2 2018-11-03       2
user_96986840

You can try alternate solution with str.contains with groupby function without using sum all along.

>>> df
         Date                                         content
0  2018-11-01  Apple Inc. AAPL 1.54% reported its fourth cons
1  2018-11-01   U.S. stocks climbed Thursday, Apple is a real
2  2018-11-02  GONE are the days when smartphone manufacturer
3  2018-11-03   To historians of technology, the story of the
4  2018-11-03  Apple Inc. AAPL 1.54% reported its fourth cons
5  2018-11-03  Apple is turning to traditional broadcasting t

Solutions:

df.content.str.contains("Apple").groupby(df['Date']).count().reset_index(name="count")

         Date  count
0  2018-11-01      2
1  2018-11-02      1
2  2018-11-03      3


# df["content"].str.contains('Apple',case=True,na=False).groupby(df['Date']).count()
user_57048630
Related