Desired
I want to capture all the winners and their opponents from the ibjjf website. This includes details like their division, belt color, gym, rank, etc. I'm doing this by scraping the branches from within each link on this page, and putting it into a dataframe. Before creating a loop, I scraped one page to test out the code I wrote, and it worked exactly as I wanted it to.
This is what the end data frame should look like:
Problems
I created a loop to capture this information from multiple pages. The division, belt, gender, and weight all only appear once on each page. However, when I ran my code on just one page, r automatically filled in the correct information for each competitor. I would like it to automatically fill in the appropriate information for all pages. For example, if the belt color is Black, I want r to fill in black next to each competitor name that appears on that page. Just like in the photo.
Here is a sample of the code I wrote for the loop:
library(rvest)
library(tidyverse)
# CREATE EMPTY VECTORS ---------------------------------------
# fight info vectors
division_all = c()
gender_all = c()
belt_all = c()
weight_all = c()
fightAndMat_all = c()
date_all = c()
competitor_all = c()
name_all = c()
gym_all = c()
# Create for loop -----------------------------------
for (i in 46:70){
MensUrl <- read_html(paste0('https://www.bjjcompsystem.com/tournaments/1869/categories/20531', i))
# SCRAPE THE DATA #############################################
## SCRAPE FIGHT INFO -------------------------------------------
# fight info
division <- MensUrl %>%
html_nodes('.category-title__age-division') %>%
html_text()
gender <- MensUrl %>%
html_nodes('.category-title__age-division+ .category-title__label') %>%
html_text()
belt <- MensUrl %>%
html_nodes('.category-title__label:nth-child(3)') %>%
html_text()
weight <- MensUrl %>%
html_nodes('.category-title__label:nth-child(4)') %>%
html_text()
fightAndMat <- MensUrl %>%
html_nodes('.bracket-match-header__where , .bracket-match-header__fight') %>%
html_text()
date = MensUrl %>%
html_nodes('.bracket-match-header__when') %>%
html_text()
competitor = MensUrl %>%
html_nodes('.match-card__competitor-n') %>%
html_text()
name = MensUrl %>%
html_nodes('.match-card__competitor-name') %>%
html_text()
gym = MensUrl %>%
html_nodes('.match-card__club-name') %>%
html_text()
# append fight info ----
division_all = append(division_all,division)
gender_all = append(gender_all,gender)
belt_all = append(belt_all,belt)
weight_all = append(weight_all, weight)
fightAndMat_all = append(fightAndMat_all, fightAndMat)
date_all = append(date_all,date)
competitor_all = append(competitor_all,competitor)
name_all = append(name_all,name)
gym_all = append(gym_all, gym)
}
# CONVERT TO DATA FRAMES --------------------------------------------------
#### create match df ####
matches = data.frame('division' = division_all,
'gender' = gender_all,
'belt' = belt_all,
'weight' = weight_all,
'fightAndMat' = fightAndMat_all,
'date' = date_all,
'competitor' = competitor_all,
'name' = name_all,
'gym' = gym_all)
> Error in data.frame(division = division_all, gender = gender_all, belt = belt_all, :
arguments imply differing number of rows: 25, 1818, 909, 2601, 2207