Scraping data from variable table with rvest

Question

I'm attempting to scrape tables from this page: https://sportsbook.draftkings.com/leagues/football/nfl?category=passing-props&subcategory=pass-tds

I'm trying to gather the info under "player","over", and "under" so the first row would be Joe Flacco 1.5 +140 1.5 -190 (these numbers change so when you're reading it might be different)

As an example of code I used on the same website, but different table/link, I used this:

library(rvest)
url <- 'https://sportsbook.draftkings.com/leagues/football/nfl?category=passing-props&subcategory=pass-yds'


test <-  url %>%
      read_html() %>%
      html_nodes('.default-color , .sportsbook-outcome-cell__line , .sportsbook-row-name') %>%
      html_text()

This code gives me the exact data that I want.

Note that this working code is for a separate page: https://sportsbook.draftkings.com/leagues/football/nfl?category=passing-props&subcategory=pass-yds

I used selectorGadget extension to ascertain the selector value

The 2 different pages I'm looking at are accessible from the header above "Bal Ravens"

Pass Yds is the default table selection for the page; Pass TDs is next to it, which gets you to the page which I'm also attempting to scrape.

For some reason, scraping the Pass TDs table using the same method as Pass Yds table leaves me with an empty string:

url <- 'https://sportsbook.draftkings.com/leagues/football/nfl?category=passing-props&subcategory=pass-tds'


test <-  url %>%
      read_html() %>%     
      html_nodes('.sportsbook-row-name , .default-color , .sportsbook-outcome-cell__line') %>%
      html_text()

Note that when using selectorGadget for this page, it gives me a different html_nodes selector

I have also tried using xpath and finding the individual tables (with html_table) on the Inspect page. Again, this process works with the Pass Yds page, but not the Pass Tds page. I assume this problem relates to the fact that the table on the website is variable, with Pass Yds being the default.

If anyone could help me with this, or point me in the direction to information regarding scraping these menu-selectable tables, I would greatly appreciate it.

Thanks!

Scraping data from variable table with rvest

0 Answers