I am trying to extract all of the tweets since a given date of a search term. However, when I run the code, I'm only getting about 1,300 tweets that end about 8 days ago. How can I get around this problem?
I used tweepy.Cursor and put my count to 1 million.
def get_tweets(listOfTweets, keyword, numOfTweets, startDate):
# Iterate through all tweets containing the given word, api search mode
for tweet in tweepy.Cursor(api.search, q=keyword, since=startDate).items(numOfTweets):
# Add tweets in this format
dict_ = {'Screen Name': tweet.user.screen_name,
'User Name': tweet.user.name,
'Tweet Created At': tweet.created_at,
'Tweet Text': tweet.text,
'User Location': tweet.user.location,
'Tweet Coordinates': tweet.coordinates,
'Retweet Count': tweet.retweet_count,
'Retweeted': tweet.retweeted,
'Phone Type': tweet.source,
'Favorite Count': tweet.favorite_count,
'Favorited': tweet.favorited,
'Replied': tweet.in_reply_to_status_id_str
}
listOfTweets.append(dict_)
return listOfTweets
get_tweets(listOfTweets=XXXXX, keyword = 'XXXXXXXXX', numOfTweets = 10000000, startDate = '2018-01-01')
I thought it would pull all the tweets but it's only returning about a week of tweets since today.
You can suggest other libraries or websites to get around this. Any help is very appreciated.