Python: How to iterate through a list of Twitter users using Snscrape?

user_15936182 0
Viewed 3530

I trying to retrieve tweets over a list of users, however in the snscrape function this argument is inside quotes, which makes the username to be taken as a fixed input

import snscrape.modules.twitter as sntwitter
tweets_list1 = []
users_name = [{'username':'@bbcmundo'},{'username':'@nytimes'}]

for i,tweet in enumerate(sntwitter.TwitterSearchScraper('from:{}').get_items().format(username)):
if i>100:
    break
tweets_list1.append([tweet.date, tweet.id, tweet.content, tweet.url,\
                     tweet.user.username, tweet.user.followersCount,tweet.replyCount,\
                    tweet.retweetCount, tweet.likeCount, tweet.quoteCount, tweet.lang,\
                    tweet.outlinks, tweet.media, tweet.retweetedTweet, tweet.quotedTweet,\
                    tweet.inReplyToTweetId, tweet.inReplyToUser, tweet.mentionedUsers,\
                     tweet.coordinates, tweet.place, tweet.hashtags, tweet.cashtags])

As output Python get:

`AttributeError: 'generator' object has no attribute 'format'

This code works fine replacing the curly braces with the username and deleting the .format attribute. If you want replicate this code be sure install snscrape library using:

pip install git+https://github.com/JustAnotherArchivist/snscrape.git

I really appreciate any guidance you can give me.

web-scraping-language scrapy twitter python
2 Answers

I found some mistakes that I did writing this code. So, I want to share with all of you just in case you need it and overcome your stuck with this very same problem or a similar one:

First: I changed the users_name format, from a dict to a list items.

Second: I put the format attribute in the right place. Right after text input function

Third: I added a nested loop to scrape each Twitter user account

users_name = ['bbcmundo','nytimes']
for n, k in enumerate(users_name):
    for i,tweet in enumerate(sntwitter.TwitterSearchScraper('from:{}'.format(users_name[n])).get_items()):
    if i>100:
        break
    tweets_list1.append([tweet.date, tweet.id, tweet.content, tweet.url,\
                         tweet.user.username, tweet.user.followersCount,tweet.replyCount,\
                        tweet.retweetCount, tweet.likeCount, tweet.quoteCount, tweet.lang,\
                        tweet.outlinks, tweet.media, tweet.retweetedTweet, tweet.quotedTweet,\
                        tweet.inReplyToTweetId, tweet.inReplyToUser, tweet.mentionedUsers,\
                         tweet.coordinates, tweet.place, tweet.hashtags, tweet.cashtags])

I hope it can be helpful

user_159361820

You can avoid to make several requests by using more than one from criteria:

users = ['bbcmundo','nytimes']
filters = ['since:2022-07-06', 'until:2022-07-07']
from_filters = []
for user in users:
    from_filters.append(f'from:{user}')
filters.append(' OR '.join(from_filters))
tweets = list(sntwitter.TwitterSearchScraper(' '.join(filters)).get_items())
# The argument is 'since:2022-07-06 until:2022-07-07 from:bbcmundo OR from:nytimes'
user_172298770
Related