Scraping Twitter Data: API Failed (Error 403)

Question

I am trying to learn how to scrape tweets from Twitter (using R). I have a twitter account and registered for the API. I got all necessary keys that seem to be required (api key, api secret, access token, access secret, bearer token).

I am now following this tutorial here: https://cran.r-project.org/web/packages/rtweet/vignettes/auth.html

I first entered this line:

auth <- rtweet_app()

And then I entered my bearer token - when I try to view this object, it looks like this:

> auth

I tried to run a query as suggested in the tutorial:

 df <- search_tweets("#rstats", token = auth)

But this returns the following error:

Error: Twitter API failed [403]
Check error message at https://developer.twitter.com/en/support/twitter-api/error-troubleshooting

I tried to search this error on Twitter, but it does not seem very informative on how I can troubleshoot this:

# https://developer.twitter.com/en/support/twitter-api/error-troubleshooting
    The request is understood, but it has been refused or access is not allowed. An accompanying error message will explain why.

Can someone please help me further troubleshoot this error?

Note: I did not get error 401 which would have indicated that I was not successfully authenticated.

Thanks!

Here is my session info:

R version 4.1.3 (2022-03-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 22000)

Matrix products: default

locale:
[1] LC_COLLATE=English_Canada.1252  LC_CTYPE=English_Canada.1252    LC_MONETARY=English_Canada.1252 LC_NUMERIC=C                   
[5] LC_TIME=English_Canada.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rtweet_1.0.2

loaded via a namespace (and not attached):
 [1] prettyunits_1.1.1 withr_2.5.0       digest_0.6.29     crayon_1.5.1      R6_2.5.1          lifecycle_1.0.1   evaluate_0.15    
 [8] httr_1.4.4        rlang_1.0.2       progress_1.2.2    cli_3.3.0         curl_4.3.2        vctrs_0.4.1       ellipsis_0.3.2   
[15] rmarkdown_2.14    tools_4.1.3       tinytex_0.40      hms_1.1.1         xfun_0.30         yaml_2.3.5        fastmap_1.1.0    
[22] compiler_4.1.3    pkgconfig_2.0.3   askpass_1.1       htmltools_0.5.2   knitr_1.39       
>

Thanks!

user_11773772 · Answer

I'm not a "R" programmer, but I found an article maybe that helps you :)

https://medium.com/analytics-vidhya/scraping-and-analyzing-tweets-in-r-62582e2f4543

user_1660210 · Answer

my_auth <- rtweet::create_token() works better but it's deprecated. But it works!

Run my_auth <- rtweet::create_token() without arguments, and you should see a login window in your browser (or confimation is you are already logged in). In a result you should get a new variable in your environment "my_auth". Like on the image:

and auth file in dir (on linux): ~/.config/R/rtweet.

Then you can make query like: obama_tweets <- rtweet::get_timeline(c("barackobama"), n = 3, parse=T, token=my_auth)

You can also use function auth_sitrep() to see which auth file is valid, because auth files created with rtweet_app() are (almost) empty (you can also check directory ~/.config/R/rtweet).

You can also reuse, saved (automatically) auth file with auth <- readRDS("~/.config/R/rtweet/default.rds").

Scraping Twitter Data: API Failed (Error 403)

2 Answers