rvest scrape in Rmarkdown knit fails even though code works without knit

user_13243824 0
Viewed 182

I run the code, it works great. I try to knit and I get this error:

Error: Can't rename columns that don't exist. The column Tests<U+2009>/millionpeople doesn't exist.

I tried, clearing cache, loading image at start, creating a new object for the rename and mutate work, and lots more. Error likely emerges because the scraped object is not being loaded (or found) during the knit, but I cant figure out why or how to fix.

Any ideas? Thanks!

My code:

library(utils) library(httr) library(tidyverse) library(rvest) library(ggpubr)

#scrapes from wikipedia, xpath is correct url <- "https://en.wikipedia.org/wiki/COVID-19_testing"  tests <- url %>%     read_html() %>%   html_nodes(xpath='//*[@id="mw-content-text"]/div/table[4]') %>%    html_table() %>%    extract2(1) %>% # extracts data table from html list   rename(country = "Country or region", tests = "Tests", positive
= "Positive", asof = "As of", 
         tests_per_million = "Tests /millionpeople" ,
         positive_per_thousand_tests = "Positive /thousandtests", ref = "Ref.") %>%   mutate(tests = as.numeric(gsub(",", "", tests)), positive = as.numeric(gsub(",", "", positive)),
         tests_per_million = as.numeric(gsub(",", "", tests_per_million)),
         positive_per_thousand_tests = round(positive_per_thousand_tests, 0)) #removes commas and coverts to numeric'
rvest r-markdown knitr r
1 Answers

There are some different names in the table with special characters that might be causing the issue. Since you want to rename all the column use rename_all. 

library(rvest)
library(dplyr)
library(readr)

url <- "https://en.wikipedia.org/wiki/COVID-19_testing"

tests <-  url %>%     
  read_html() %>%
  html_nodes(xpath='//*[@id="mw-content-text"]/div/table[4]') %>%
  html_table() %>%
  .[[1]] %>%
  rename_all(~c("country", "tests", "positive", "asof", 
                "tests_per_million","positive_per_thousand_tests", "ref")) %>%   
   mutate(tests = parse_number(tests), positive = parse_number(positive),
          tests_per_million = parse_number(tests_per_million),
          positive_per_thousand_tests = round(positive_per_thousand_tests))
user_39629140
Related