Tag found but class not visible Python Selenium

user_8333729 0
Viewed 80

I've spent over 4 hours and counting trying to get an element that's visible on the web page but won't pull when I try using Selenium to scrape it. I even tried to copy direct xpath to element but still does not work. Inspecting the page on my browser shows the structure to be like this.

<div class="flex-deprecated flex-items-xs-center">
  <div>...<div>
  <div style="position:relative">
    <div class="row flex-xs-nowrap no-margin form-left form-box-container ">
      <div class="form-box flex-xs-no-grow cursor-pointer ">
        <svg class="margin-center">
          <rect class="form-in-shape"></rect>
          <text class="form-win-text">TEXT</text>
        </svg>
      </div>
      <div class="form-box flex-xs-no-grow cursor-pointer ">...</div>
      <div class="form-box flex-xs-no-grow cursor-pointer ">...</div>
      <div class="form-box flex-xs-no-grow cursor-pointer ">...</div>
      <div class="form-box flex-xs-no-grow cursor-pointer ">...</div>
      <div class="row flex-items-xs-middle form-arrow-line no-margin left">...</div>
    </div>
 </div>

TEXT is what I'm scraping for, I was able to locate upto all the 5 div tags with class="form-box flex-xs-no-grow cursor-pointer ", but it won't find the svg tag or the text tag, when I try using the direct xpath, it finds up to the text tag but returns empty string with I try to get the text within the tag. Here is my code as I am currently out of ideas on how to approach this.

z = driver.find_element_by_xpath("//div[@class='sub-container head-to-head']/div[3]/div/div[@class='panel margin-bottom ']/div/div[@class='row']")
        
        z1 = z.find_element_by_xpath(".//div[@class='col-xs-6']/div[@class='row']/div[@class='col-xs-12 col-sm-5 col-md-7 flex-deprecated flex-items-xs-middle flex-items-xs-center']/div/div[@class='visible-md-up hidden-xs-up']/div/div[@class='row']/div/div/div[2]/div[@class='row flex-xs-nowrap no-margin form-left form-box-container ']")
        z2 = z1.find_elements_by_xpath(".//div[@class='form-box flex-xs-no-grow cursor-pointer ']")
        
        try:
            for i_ in range(0, 5):
                ztemp1 = z2[i_].find_element_by_class_name("margin-center")
                wait.until(EC.presence_of_element_located((By.CLASS_NAME, "form-with-text")))
                #ztemp = ztemp1.find_elements_by_class_name("form-with-text")
                ztemp =ztemp1.find_element_by_tag_name("text")
                #print("TESTIN>>> {}".format(len(ztemp)))
                print("SKKKKKK>>>>> {}".format(ztemp.text))
                z3 = z2[i_].find_elements_by_xpath(".//*")
                
                print('good here {}'.format(len(z3)))
                
        except(NoSuchElementException):
            print('Tag or Class not found')

When I try to find the text tag by class name, it does not find it, but it finds it when I search by tag name but it's an empty string. Here's the link to the site I'm using [Source Site][1]

Any help and tip will be highly appreciated. [1]: https://s5.sir.sportradar.com/betking/en/1/season/83706/headtohead/17/42/match/27751100

selenium-webdriver web-scraping selenium python-3.x
2 Answers

That is a svg element. You can locate like this :

//*[name()='svg' and @class='margin-center']//*[name()='text']

this is giving me 66 web elements.

you can loop through it like below :

driver = webdriver.Chrome(driver_path)
driver.maximize_window()
driver.implicitly_wait(50)
driver.get("https://s5.sir.sportradar.com/betking/en/1/season/83706/headtohead/17/42/match/27751100")

for txt in driver.find_elements(By.XPATH, "//*[name()='svg' and @class='margin-center']//*[name()='text']"):
  print(txt.text)
user_94440730

You can also try this:

driver.implicitly_wait(10)
driver.get("https://s5.sir.sportradar.com/betking/en/1/season/83706/headtohead/17/42/match/27751100")
time.sleep(5)
elements = driver.find_elements_by_xpath("//div[contains(@class,'col-sm-5')]//div[contains(@class,'hidden-md-up')]")
for ele in elements:
    datalist = []
    data = ele.find_elements_by_xpath(".//div[contains(@class,'cursor-pointer')]")
    for d in data:
        datalist.append(d.get_attribute("innerText"))
    print(datalist)

Output:

['W', 'L', 'L', 'W', 'W']
['L', 'L', 'L', 'L', 'D']
user_164528400
Related