rvest html_table() error max(p) returning -Inf

user_15148261 0
Viewed 93

I'm trying to scrape a table from the web (here https://www.cryptoslam.io/nba-top-shot/marketplace).

I have been researching how to do this and seem to have gotten closest using library rvest and the html_table() function. In fact I was able to download the "FIFA World Cup Record" table from here https://en.wikipedia.org/wiki/Brazil_national_football_team using the code

webpage_url <- "https://en.wikipedia.org/wiki/Brazil_national_football_team"
webpage <- xml2::read_html(webpage_url)
tbls <- html_nodes(webpage, "table")
head(tbls)
tbls_ls <- webpage %>%
  html_nodes("table") %>%
  .[[6]] %>%
  html_table(fill = TRUE)

Note that I have the libraries library(xml2), library(rvest) loaded. I then am using essentially the same code here:

webpage_url <- "https://www.cryptoslam.io/nba-top-shot/marketplace"
webpage <- xml2::read_html(webpage_url)
tbls <- html_nodes(webpage, "table")
head(tbls)
tbls_ls <- webpage %>%
  html_nodes("table") %>%
  .[[1]] %>%
  html_table(fill = TRUE)

but getting the error

Error in matrix(NA_character_, nrow = n, ncol = maxp) : 
  invalid 'ncol' value (too large or NA)
In addition: Warning messages:
1: In max(p) : no non-missing arguments to max; returning -Inf
2: In matrix(NA_character_, nrow = n, ncol = maxp) :
  NAs introduced by coercion to integer range

I have not been able to find any discussion of this error anywhere else. One thing that is different between the two tables is the existence of a thead tag in the second one which won't work. I have quite limited knowledge of html so I may be missing some other important differences between the table implementations.

rvest r html javascript
2 Answers

One approach is with RSelenium:

library(RSelenium)
library(rvest) #requires xml2, no need to load separately 
driver <- rsDriver(browser="chrome", port=4234L, chromever ="87.0.4280.87")
client <- driver[["client"]]
client$navigate(webpage_url)
source <- client$getPageSource()[[1]]
  
read_html(source) %>% 
  html_nodes("table") %>%
  html_table() %>%
  `[[`(1) -> result

head(result)
            Listed  Rank                   Crypto        Set                  Team Play Category   SN# Current Price          Owner
1 NA 5 minutes ago 10324     2020-21 Bradley Beal   Base Set    Washington Wizards       Handles 10691   (10.00 USD)        P1BenEe
2 NA 5 minutes ago  1096     2019-20 Kelly Olynyk The Finals            Miami Heat         Layup   360  (180.00 USD) Top_Shot3point
3 NA 5 minutes ago  3138      2019-20 Alex Caruso   Base Set    Los Angeles Lakers         Block   679     67.00 USD CaptainThunder
4 NA 5 minutes ago  3586  2020-21 Kelly Oubre Jr.   Base Set Golden State Warriors          Dunk  3583      5.00 USD       dddd9999
5 NA 5 minutes ago  3318  2020-21 Bismack Biyombo   Base Set     Charlotte Hornets         Layup  3315      7.00 USD       ectoasty
6 NA 5 minutes ago  4940 2020-21 DeMarcus Cousins   Base Set       Houston Rockets     3 Pointer  4937    (3.00 USD) StoneColdBroke
user_130953260

That data comes from an API post request returning json. You can issue that request then parse json, returned as list from content, into whatever your desired format is

library(httr)

headers = c(
  'user-agent'= 'Mozilla/5.0',
  'content-type'= 'application/json',
  'referer'= 'https://www.cryptoslam.io/',
  'accept-language'= 'en-GB,en-US;q=0.9,en;q=0.8'
)

data = '{"draw":1,"columns":[{"data":null,"name":"","searchable":true,"orderable":false,"search":{"value":"","regex":false}},{"data":null,"name":"TimeStamp","searchable":true,"orderable":true,"search":{"value":"","regex":false}},{"data":null,"name":"","searchable":true,"orderable":false,"search":{"value":"","regex":false}},{"data":null,"name":"Tokens.0.Rank","searchable":true,"orderable":true,"search":{"value":"","regex":false}},{"data":null,"name":"Tokens.Attributes.Name","searchable":true,"orderable":true,"search":{"value":"","regex":false}},{"data":null,"name":"Tokens.Attributes.Set","searchable":true,"orderable":false,"search":{"value":"","regex":false}},{"data":null,"name":"Tokens.Attributes.Team","searchable":true,"orderable":false,"search":{"value":"","regex":false}},{"data":null,"name":"Tokens.Attributes.PlayCategory","searchable":true,"orderable":false,"search":{"value":"","regex":false}},{"data":null,"name":"Tokens.Attributes.SerialNumber","searchable":true,"orderable":true,"search":{"value":"","regex":false}},{"data":null,"name":"CurrentPrice","searchable":true,"orderable":true,"search":{"value":"","regex":false}},{"data":null,"name":"EndingPriceGwei","searchable":true,"orderable":true,"search":{"value":"","regex":false}},{"data":null,"name":"","searchable":true,"orderable":false,"search":{"value":"","regex":false}},{"data":null,"name":"","searchable":true,"orderable":false,"search":{"value":"","regex":false}}],"order":[{"column":1,"dir":"desc"}],"start":0,"length":50,"search":{"value":"","regex":false},"startdate":"","enddate":"","marketplace":"","attributesQuery":{}}'

r <- httr::POST(url = 'https://api2.cryptoslam.io/api/marketplace/NBA Top Shot/search', httr::add_headers(.headers=headers), body = data) %>% content()
user_62412350
Related