How to get text in td web-table with Selinium Python

user_10918819 0
Viewed 886

I am trying to extract text from a td webtable but have not been able to locate the element; the table has no class or id so I have attempted xpath without success.

Any help is greatly appreciated. 

driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get("https://www.ncbi.nlm.nih.gov/tools/primer-blast/primertool.cgi?ctg_time=1585700551&job_key=9P4rCho2F54woA2lAMUpl3reOKVXzSO4Vg&CheckStatus=Check")

pair_1 = driver.find_element(By.XPATH("html/body/div[@id ='wrap']/div[@id='content-wrap']/div[@id='content']/div[contains(@class, ' ')]/div[contains(@class, ' ')]/div[contains(@class, 'ui-helper-resert')]/div[@id ='alignInfo']/div[@id ='alignments']/table/tbody/tr[2]/td[1]"))

print(pair_1.text)

#OR

pair_1.get_attribute("innerHTML")

print(pair_1)

Return the following error

TypeError: 'str' object is not callable

A simpler xpath

pair_1 = driver.find_element_by_xpath("//table/tbody/tr[2]/td[1]")
print(pair_1.text)

Returns this

Looking for [chromedriver 80.0.3987.06 mac6] driver in cache 
File found in cache by path [/Users/usr/.wdm/drivers/chromedriver/80.0.3987.06/mac6/chromedriver]

website and html enter image description here enter image description here

Thank you advance, I am a novice with Selenium.

selenium-webdriver selenium html-table html python
3 Answers

Try the following

import time

from selenium import webdriver

driver = webdriver.Chrome()
driver.get("https://www.ncbi.nlm.nih.gov/tools/primer-blast/primertool.cgi?ctg_time=1585700551&job_key=9P4rCho2F54woA2lAMUpl3reOKVXzSO4Vg&CheckStatus=Check")
time.sleep(2)

pair_1 = driver.find_element_by_css_selector(
    "#alignments > div:nth-child(1) > table > tbody > tr:nth-child(2) > td:nth-child(2)"
).get_attribute("innerHTML")


print(pair_1)

on windows, chrome Version 80

select the text > Right click > inspect > Right click > inspect > right click on <td> copy selector

enter image description here

enter image description here

user_45397090

Try following code:

driver.get('https://www.ncbi.nlm.nih.gov/tools/primer-blast/primertool.cgi?ctg_time=1585700551&job_key=9P4rCho2F54woA2lAMUpl3reOKVXzSO4Vg&CheckStatus=Check')

str1 = "Primer pair 1";
str2 = "Forward primer";
el = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//h2[text()='" +str1 +"']//following-sibling::table//th[text()='" +str2+"']//following-sibling::td[1]")))
print(el.text)

You can change string value.

And this error TypeError: 'str' object is not callable appear because you wrong in this line:

pair_1 = driver.find_element(By.XPATH("the value"))

It should:

pair_1 = driver.find_element(By.XPATH, "the value")
user_87520620

To get the value of 'Forward primer','Reverse primer'and 'Product length' from the Primer pair 1 Induce WebDriverWait() and visibility_of_element_located() and following XPath.

1.Create a list of items and then iterate.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

product_list=['Forward primer','Reverse primer','Product length']
driver=webdriver.Chrome()
driver.get("https://www.ncbi.nlm.nih.gov/tools/primer-blast/primertool.cgi?ctg_time=1585700551&job_key=9P4rCho2F54woA2lAMUpl3reOKVXzSO4Vg&CheckStatus=Check")
for item in product_list:
    print(WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//h2[text()='Primer pair 1']/following::table[1]//th[text()='{}']/following-sibling::td[1]".format(item)))).text)

Output:

GAGGGCATACCCCTCGTAGA
CTGCCGTTTTCCGTAGGACT
506

To get data from All Primer pairs try this.

1.Create a list of primers and then iterate.

2.Create a list of items and then iterate.

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

driver=webdriver.Chrome()
driver.get("https://www.ncbi.nlm.nih.gov/tools/primer-blast/primertool.cgi?ctg_time=1585700551&job_key=9P4rCho2F54woA2lAMUpl3reOKVXzSO4Vg&CheckStatus=Check")

list_primerpairs=['Primer pair 1','Primer pair 2','Primer pair 3','Primer pair 4','Primer pair 5']
product_list=['Forward primer','Reverse primer','Product length']
for primer in list_primerpairs:
    print(primer)
    print("===========================================")
    for item in product_list:

        print(WebDriverWait(driver,15).until(EC.visibility_of_element_located((By.XPATH,"//h2[text()='{}']/following::table[1]//th[text()='{}']/following-sibling::td[1]".format(primer,item)))).text)
    print("==========================================")

Output:

Primer pair 1
===========================================
GAGGGCATACCCCTCGTAGA
CTGCCGTTTTCCGTAGGACT
506
==========================================
Primer pair 2
===========================================
AGTCCTACGGAAAACGGCAG
GCTATTCTCGCAGCTCACCA
621
==========================================
Primer pair 3
===========================================
AGGTAGTCAGTCAGGTCCCG
TGCCGTTTTCCGTAGGACTC
558
==========================================
Primer pair 4
===========================================
AGGGCATACCCCTCGTAGAT
TCTGCCGTTTTCCGTAGGAC
506
==========================================
Primer pair 5
===========================================
GGGCATACCCCTCGTAGATG
GCCGTTTTCCGTAGGACTCT
502
==========================================

Hope this helps.

user_108856840
Related