Calculate Tweets Per day

Question

I need to write a function which calculates the number of tweets that were posted per day.

Function Specifications:

It should take a pandas dataframe as input. It should return a new dataframe, grouped by day, with the number of tweets for that day.
The index of the new dataframe should be named Date, and the column of the new dataframe should be 'Tweets', corresponding to the date and number of tweets, respectively.
The date should be formated as yyyy-mm-dd, and should be a datetime object.

My code is :


def number_of_tweets_per_day(df):
    
    df = pd.DataFrame(twitter_df['Date','Tweets'], columns =['Date', 'Tweets'])
    
    df['Date'] = pd.to_datetime(df['Date'], format = '%Y%m%d')
    df['Tweets'] = df['Tweets'].astype(str)
    
    grouped = pd.df.groupby(['Date'],['Tweets'])
    final = pd.df.count()
    
    
    
    return final

However when I try to run command 'number_of_tweets_per_day(twitter_df.copy())' I get an error saying twitter_df not defined.

DF: DataFrame

Expected output:

Attached links to an image of input dataframe and image of expected output.

What am I doing wrong?

user_12416453 · Answer

You can use DataFrame.groupby with pd.Series.dt.date to group and use GroupBy.count here.

def tweets_per_day(df):
    df['Date'] = pd.to_datetime(df['Date'], format='%Y%m%d')
    return df[['Tweets']].groupby(df['Date'].dt.date).count()
  # return df['Tweets'].groupby(df['Date'].dt.date).count() 
  # if you want output to be `Series`

tweets_per_day(twitter_df)

Output on dummy df:

d = pd.to_datetime(['2019-11-29', '2019-11-30']).repeat(10)
df = pd.DataFrame({'Tweets':'anything', 'Date':d})
df
#      Tweets       Date
#0   anything 2019-11-29
#1   anything 2019-11-30
#2   anything 2019-11-29
#3   anything 2019-11-30
#4   anything 2019-11-29
#5   anything 2019-11-30
#6   anything 2019-11-29
#7   anything 2019-11-30
#8   anything 2019-11-29
#9   anything 2019-11-30
#10  anything 2019-11-29
#11  anything 2019-11-30
#12  anything 2019-11-29
#13  anything 2019-11-30
#14  anything 2019-11-29
#15  anything 2019-11-30
#16  anything 2019-11-29
#17  anything 2019-11-30
#18  anything 2019-11-29
#19  anything 2019-11-30

tweets_per_day(df)

            Tweets
Date
2019-11-29      10
2019-11-30      10

user_13608599 · Answer

The groupby() technique demonstrated by @Ch3steR certainly works. Here are two other approaches, using value_counts() and resample():

Create the data frame (and set data type):

import pandas as pd

df = pd.DataFrame({'Date':  ['2019-11-30'] * 5 + ['2019-11-29'] * 6,
                   'Tweets': 'x'}
                 )
df['Date'] = pd.to_datetime(df['Date'], format='%Y-%m-%d')

The value_counts() approach:

def tweets_per_day_2(df):
    return df['Date'].value_counts().sort_index()

result = tweets_per_day_2(df)
print(result)

2019-11-29    6
2019-11-30    5
Name: Date, dtype: int64

And the resample() approach:

def tweets_per_day_3(df):
    return df.set_index('Date').resample('D').count().sort_index()

result = tweets_per_day_3(df)
print(result)

            Tweets
Date              
2019-11-29       6
2019-11-30       5

Calculate Tweets Per day

2 Answers