Character(0) error when using rvest to webscrape Google search results

user_14469548 0
Viewed 296

I am trying to scrape the headers of the Google search. However, doesn't matter what I try with rvest, the result always returns character(0).

Here is the code for the search rstudio:

library(rvest)
library(dplyr)

web1 <- read_html("https://www.google.at/search?q=rstudio") 
header <-web1 %>%
    html_nodes(".DKV0Md") %>%
    html_text()
header

The node name I checked in SelectorGadget, so that shouldn't be an issue. How should I tackle this problem?

rvest web-scraping r google-search
1 Answers

Perhaps, we can use:

library(rvest)
library(dplyr)
web1 %>% 
   html_nodes(xpath = '//div/div/div/a/div[not(div)]') %>% 
   html_text

Output:

#[1] "rstudio.com"        
#[2] "rstudio.cloud"           
#[3] "en.wikipedia.org › wiki › RStudio"    
# ....
user_37322710
Related