Is there a way to aggregate the number of times a keyword is mentioned in a Subreddit over time?

user_7202229 0
Viewed 615
import requests
import pandas as pd

def get_pushshift_data(data_type, **kwargs):
    
    """
    Gets data from the pushshift api.

    """
 
    base_url = f"https://api.pushshift.io/reddit/search/{data_type}/"
    payload = kwargs
    request = requests.get(base_url, params=payload)
    return request.json()

#paramaters#
data_type="comment" 
query="chicken"       
duration="30d"         
size=1000                        
aggs="subreddit"        


data = get_pushshift_data(data_type=data_type,
                          q=query,
                          after=duration,
                          size=size,
                          aggs=aggs)

data = data.get("aggs").get(aggs)

print(data[1])

This script produces a 'doc_count' and a 'key' variable, representing the number of comments ('doc_count') that appear within a Subreddit ('key') that mention our defined query term over or defined time-period (duration).

Is there any way to add a time variable to this data so that we can see the top 20 Subreddits that our query term appeared in each day for a defined duration of time?

I'm ultimately trying to see the change in Subreddit activity over time for our query term (so maybe I could incorporate this into a visualization that has a time animation etc.).

reddit time-series python
1 Answers

You can use after and before param as mentioned in documentation https://pushshift.io/api-parameters/

for example:

import requests
import pandas as pd
import time

def get_pushshift_data(data_type, **kwargs):

    """
    Gets data from the pushshift api.

    """

    base_url = f"https://api.pushshift.io/reddit/search/{data_type}/"
    payload = kwargs
    request = requests.get(base_url, params=payload)
    return request.json()

#paramaters#
data_type="comment"
query="chicken"
# duration="30d"
size=1000
aggs="subreddit"

after = int(time.time()) - 60 * 60 * 24 * 2
before = int(time.time()) - 60 * 60 * 24


data = get_pushshift_data(data_type=data_type,
                          q=query,
                          size=size,
                          aggs=aggs,
                          after=after,
                          before=before)

data = data.get("aggs").get(aggs)

print(data[1])
user_108915920
Related