Web scraping a table from Tennis 24

Question

Hi I am looking to periodically scrape the values from a table such as this one https://www.tennis24.com/match/Wra2Ija2/#match-statistics;0. In an ideal world I'd like to scrape when there's been a change to the page. (I don't know wether this is even possible to do.)

I'm thinking to check every 3 minutes. Is this a good idea? Or is there a simpler approach?

Also, here's my code:
It only pulls player Bs Aces though...

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Chrom_path = r"C:\Users\Dan1\Desktop\chromedriver.exe"
driver = webdriver.Chrome(Chrom_path)
driver.get("https://www.tennis24.com/match/hOYDXnLI/#match-statistics;0")
print(WebDriverWait(driver, 
20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='statText 
statText--titleValue' and 
text()='Aces']//following::div"))).get_attribute("innerHTML"))

user_14196152 · Answer

The data in the table has an iterated pattern with the tag

which contains 3 child nodes :

0

Aces

7

respectively for the home player data, the label of the data, and the away player data.

My following script iterates over these nodes and print inner text content :

from selenium import webdriver

driver = webdriver.Chrome("../chromedriver")
driver.get("https://www.tennis24.com/match/hOYDXnLI/#match-statistics;0")

data = driver.find_elements_by_class_name("statTextGroup")
for d in data:
    sub_data = d.find_elements_by_xpath(".//*")
    assert len(sub_data)==3
    for s_d in sub_data:
        print(s_d.get_attribute('class')[19:], s_d.get_attribute('innerText'))

driver.close()

Output displayed looks like :

homeValue 0
titleValue Aces
awayValue 3
homeValue 1
titleValue Double Faults
awayValue 0
homeValue 58%
titleValue 1st Serve Percentage
awayValue 62%
homeValue 60% (9/15)
titleValue 1st Serve Points Won
awayValue 45% (15/33)
homeValue 73% (8/11)
titleValue 2nd Serve Points Won

Note that these data pattern is repeated 4 times (for your example) for whole game data, set 1, set 2 and set 3

Pay attention when the data labeled "Aces" is repeated

Web scraping a table from Tennis 24

1 Answers