I am trying to scrape the date from the "See More" section of this website
I have located the specific div for the See More actuator:
div _ngcontent-flo-app-c111
but cannot access any of the content under it. What am I missing here?
1 Answers
The data you see after clicking on See More is embedded inside the page in the form of Json. To parse it, you can use next example:
import json
import requests
from bs4 import BeautifulSoup
id_num = "7417030" # ID is from URL
url = "https://tv.varsity.com/results/7417030-2022-spirit-sports-west-palm-beach-nationals-didii/31606"
soup = BeautifulSoup(requests.get(url).content, "html.parser")
x = soup.select_one("#flo-app-state").contents[0]
x = (
x.replace("&q;", '"')
.replace("&l;", "<")
.replace("&g;", ">")
.replace("&a;", "&")
)
x = json.loads(x)[
f"G.https://tv-api.varsity.com/api/result-containers/{id_num}?"
]
soup = BeautifulSoup(
x["body"]["data"]["event_node"]["description"], "html.parser"
)
print(soup.get_text(separator="\n"))
Prints:
Welcome to the 2022 Spirit Sports West Palm Beach Nationals DI/DII event hub! Click 'Read More' below to find the very best coverage of the competition including a live stream, the order of competition, results, photos, articles, news, and more!
Live Stream Information
Detailed coverage information will be posted as it becomes available.
Live Stream Availability
Due to music rights, live streams are available in USA, UK, Canada and Puerto Rico only. However, all other content, including on-demand competition routine videos, original documentaries and articles are available everywhere.
Championship Schedule
Saturday, February 12th
8:00 AM - 7:47 PM
Sunday, February 13th
8:00 AM - 4:52 PM
*Schedule is subject to change.
Competition Routine Videos
...and so on.
Related