How can I scrape the following data using RVEST?

Shehroz Malik 0
Viewed 25

How can I scrape the data on this website - all the RVEST resources i am referring to are not helping. The website is: https://www.achc.org/find-a-provider/

I need the PCAB Compounding Pharmacies in the United States. I am using the following code

#Specifying the url for desired website to be scraped
url <- 'https://www.achc.org/find-a-provider/'

#Reading the HTML code from the website
read_html(url) %>% html_node('.company_name')

Thank you for your help

rvest
1 Answers

The page does an ajax request returning json. Within that json is the html with the data of interest. You can mimic that request passing in the parameters you can find in the network tab of a browser when manually selecting the desired option on the webpage.

library(httr2)
library(rvest)
library(stringr)

url <- "https://www.achc.org/wp-admin/admin-ajax.php"
headers <- c("user-agent" = "mozilla/5.0")

r <- request(url) |>
  (\(x) req_headers(x, !!!headers))() |>
  req_url_query(action = "filter_provider_data", provider_id = 11) |>
  req_perform() |>
  resp_body_json(check_type = FALSE)

page <- r$response_html |> read_html()

listings <- map(page |> html_elements(".list_cont_box"), ~ .x |>
  html_text(trim = T) |>
  str_replace_all("[\\n\\t]+", ", "))
QHarr0
Related