How can I get around Tweepy's limits for tweet extraction?

Viewed 320

I am trying to extract all of the tweets since a given date of a search term. However, when I run the code, I'm only getting about 1,300 tweets that end about 8 days ago. How can I get around this problem?

I used tweepy.Cursor and put my count to 1 million.

def get_tweets(listOfTweets, keyword, numOfTweets, startDate):
    # Iterate through all tweets containing the given word, api search mode
    for tweet in tweepy.Cursor(api.search, q=keyword, since=startDate).items(numOfTweets):
        # Add tweets in this format
        dict_ = {'Screen Name': tweet.user.screen_name,
                'User Name': tweet.user.name,
                'Tweet Created At': tweet.created_at,
                'Tweet Text': tweet.text,
                'User Location': tweet.user.location,
                'Tweet Coordinates': tweet.coordinates,
                'Retweet Count': tweet.retweet_count,
                'Retweeted': tweet.retweeted,
                'Phone Type': tweet.source,
                'Favorite Count': tweet.favorite_count,
                'Favorited': tweet.favorited,
                'Replied': tweet.in_reply_to_status_id_str
                }
        listOfTweets.append(dict_)

    return listOfTweets

get_tweets(listOfTweets=XXXXX, keyword = 'XXXXXXXXX', numOfTweets = 10000000, startDate = '2018-01-01')

I thought it would pull all the tweets but it's only returning about a week of tweets since today.

You can suggest other libraries or websites to get around this. Any help is very appreciated.

0 Answers
Related