Unable to obtain table info through python selenium

user_7747401 0
Viewed 48

I am new bee on python selenium environment. I am trying to get the SQL version table from enter link description here

from selenium.webdriver.common.by import By
from selenium import webdriver
# define the website to scrape and path where the chromediver is located
website = "https://www.sqlserverversions.com"
driver = webdriver.Chrome(executable_path='/Users//Downloads/chromedriver/chromedriver.exe')
# define 'driver' variable
# open Google Chrome with chromedriver
driver.get(website)
matches = driver.find_elements(By.TAG_NAME, 'tr')
for match in matches:
b=match.find_elements(By.XPATH,"./td[1]")
print(b.text)

it says AttributeError: 'list' object has no attribute 'text'. Am i choosing the write syntax and right parameters to grab the data?

Below is the table which i am trying to get data. enter image description here

Below are the parameters which i am trying to put in code.

enter image description here

Please advise what is required to modify in the code to obtain the data in table format.

Thanks, Arun

web-scraping selenium python
3 Answers

There was a part of your code where you were calling b.text after getting the result of find_elements, which returns a list. You can only call b.text on a single WebElement (not a list of them). Here's the updated code:

from selenium.webdriver.common.by import By
from selenium import webdriver

website = "https://www.sqlserverversions.com"
driver = webdriver.Chrome(executable_path='/Users//Downloads/chromedriver/chromedriver.exe')
driver.get(website)
matches = driver.find_elements("css selector", "tr")
for match in matches[1:]:
    items = match.find_elements("css selector", "td")
    for item in items:
        print(item.text)

That will print out A LOT of rows, unless you limit the loop.

user_70582660

If you need data only from first table:

from selenium.webdriver.common.by import By
from selenium import webdriver

website = "https://www.sqlserverversions.com"
driver = webdriver.Chrome(executable_path='/Users//Downloads/chromedriver/chromedriver.exe')
driver.get(website)

show_service_pack_versions = True
xpath_first_table_sql_rows = "(//table[@class='tbl'])[1]//tr/td/a[starts-with(text(),'SQL Server')]//ancestor::tr"

matches = driver.find_elements(By.XPATH, xpath_first_table_sql_rows)
for match in matches:
    sql_server_a_element = match.find_element(By.XPATH, "./td/a[2]")
    print(sql_server_a_element.text)

    sql_server_rtm_version_a_element = match.find_element(By.XPATH, ".//td[@class='rtm']")
    print('RTMs:')
    print(sql_server_rtm_version_a_element.text)

    if(show_service_pack_versions):
        print('SPs:')
        sql_server_sp_version_td_elements = match.find_elements(By.XPATH, ".//td[@class='sp']")
        for td in sql_server_sp_version_td_elements:
            print('---')
            print(td.text)

    print('----------------------------------')

if you set show_service_pack_versions = False then information regarding service packs will be skipped

user_43936660

If you just need text it's simpler to do it on the browser side:

data = driver.execute_script("""
  return [...document.querySelectorAll('tr')].map(tr => [...tr.querySelectorAll('td')].map(td => td.innerText))
""")
user_9660230
Related