Extraction of links instead of characters within each cell in a html_table using R

user_13655050 0
Viewed 123

I tried to extract multiple html_table using rvest package in R using the scripts:

library(rvest)
library(dplyr)
library('xml2')
library(tidyverse)
jump <- seq(1, 2, by = 1)
urls <- paste('https://asbdavani.org/horse/foals/', jump, sep="")
out <- vector("character", length = length(urls))
for(i in seq_along(urls)){
  derby <- read_html(urls[i], encoding="UTF-8")
  out[i] <- derby %>%
    html_table(fill = TRUE)
}
first_table <- out[[1]]

Here, I extracted one of those tables as first_table: enter image description here

I want to know how can I have links of each character in columns 2, 6, and 7 like this : enter image description here

xml2 rvest dplyr tidyverse r
2 Answers

This is a tricky problem as you need to select specific columns within a table. The xpath selector "nth-child" provides that ability.
The code below will demonstrate the solution on just 1 table from 1 page in order to simplify the explanation. It should be relative easy to copy and paste into your code.

#Read the page
url<-"https://asbdavani.org/horse/foals/6404"
page <- read_html(url)

#extract the tables from the page
tables <-page %>% html_elements("table")

#In this case we are looking at the second table
#extract each row of the table
rows <-tables[2] %>% html_elements("tr") 

#remove the first row since that is the heading
#get the 2nd column from each row
#and parse the "a" html tag from the 2nd column
#retrieve the href link
col2Links <- rows[-1] %>% html_element("td:nth-child(2) a") %>% html_attr("href")

#repeat for columns 6 & 7
col6Links <- rows[-1]%>% html_element("td:nth-child(6) a") %>% html_attr("href")
col7Links <- rows[-1]%>% html_element("td:nth-child(7) a") %>% html_attr("href")

#will need to paste0 "https://asbdavani.org" onto each link.
#col2Links %>% paste0("https://asbdavani.org", .)

#make data.frame
answer <-data.frame(col2Links, col6Links, col7Links)
answer

                  col2Links               col6Links                col7Links
1  /horse/performance/13993 /horse/performance/6404                     <NA>
2  /horse/performance/13873 /horse/performance/6404                     <NA>
3    /horse/performance/533 /horse/performance/6404 /horse/performance/10958
4   /horse/performance/5277 /horse/performance/6404 /horse/performance/11051
5   /horse/performance/5461 /horse/performance/6404 /horse/performance/11049
6   /horse/performance/5602 /horse/performance/6404 /horse/performance/11084
7   /horse/performance/6466 /horse/performance/6404 /horse/performance/11097
8  /horse/performance/11004 /horse/performance/6404 /horse/performance/10994
9  /horse/performance/11113 /horse/performance/6404 /horse/performance/11097
10 /horse/performance/11114 /horse/performance/6404 /horse/performance/11097
11 /horse/performance/11126 /horse/performance/6404 /horse/performance/11119
user_57922440

This is a rather untidy implementation, but works in principle. You can surely make it more concise and I did not relocate and rename the columns according to your exact example.

library(rvest)
library(dplyr)
library('xml2')
library(tidyverse)
jump <- seq(1, 2, by = 1)
urls <- paste('https://asbdavani.org/horse/foals/', jump, sep="")
out <- data.frame()
for(i in seq_along(urls)) {
  html <- read_html(urls[i], encoding = "UTF-8")
  derby <-
    html %>% html_elements("td") %>%
    html_children() %>%
    html_attr('href')
  links <-
    matrix(derby,
           nrow = length(derby) / 3,
           ncol = 3,
           byrow = T) %>% as.data.frame()
  combined <- html %>%
    html_table(fill = TRUE) %>% bind_cols(., tibble(
      اسب  = links$V1,
      سیلمی  = links$V2,
      مادیان  = links$V3
    ))
  
  out <- bind_rows(out, combined)
}
user_124882170
Related