Remove multiple header rows from table with Rvest in R

user_7466454 0
Viewed 201

I'm trying to scrape a table from Sports Reference:

cu_url <- "https://www.sports-reference.com/cbb/schools/creighton/"

I was able to get the table into a data frame as intended like this:

cu_html <- read_html(cu_url)
cu_table <- html_nodes(cu_html, "table")
cu_info <- data.frame(html_table(cu_table))
colnames(cu_info) <- cu_info[1,]
cu_info <- cu_info[-1,]

However, I noticed after the fact that the header row repeats throughout the data. For example, row 22 shows the headers again as a row. Is there an efficient way to remove these? In the HTML, the header rows all have a table row () class of "thead" so I'm wondering if I can ask rvest to ignore these but I've failed when trying this using ! =.

Appreciate any thoughts. If I need to remove the actual header in order for this to work I will but would prefer to keep that one and just remove the repeats.

rvest dplyr web-scraping r
1 Answers

You can keep only the rows which have only numbers in Rk column.

library(rvest)
library(dplyr)

cu_url %>%
  read_html %>%
  html_nodes('table') %>%
  html_table() %>%
  .[[1]]  %>%
  setNames(make.unique(unlist(.[1,]))) %>%
  slice(-1L) %>%
  filter(grepl('^\\d+$', Rk)) -> result

result
user_39629140
Related