How can I search a specific element on the page for text using XPATH (Selenium);

user_14681735 0
Viewed 121

I can't seem to find an example of this.

What I am trying to do is search a specific div element on the page for text that has the potential to change.

So it'd be like this

<div id="coolId">
   <div>This</div>
   <div>Can</div>
   <div>Change depending on the iteration of the page</div>
</div>

In my case, the div coolID will always be present, but the text within it's inner divs and child elements will change depending on which iteration of the page is loaded, and I need to search for the presence of certain terms within this coolID div and cool div only because I know it will always be there, and I'd like to specify the search as much as possible so as not to potentially contaminate results with other text from other places on the page.

In my head, I sort of see it like this (using the above example):

"//div[@id='coolId', contains(text(), 'Change depending on the iteration of the page')]"

Or something to this effect.

Does anyone know how to do this?

web-scraping selenium xpath python
3 Answers

You should try:

div_element_with_needed_text = driver.find_element_by_xpath("//div[@id='coolId']/div[text()[contains(.,'Change depending on the iteration of the page')]]")
user_175561720

I'm not completely sure you can set a correct XPath based on all 3 inner elements texts.
What you clearly can is to locate the outer div with id = coolId based on one of the inner texts that will be unique and then to extract all the inner texts form it.

the_total_text = driver.find_element_by_xpath("//div[@id and contains(.,'Change depending on the iteration of the page')]").text

This will give you

the_total_text = This Can Change depending on the iteration of the page
user_34854340

Considering the HTML:

<div id="coolId">
   <div>This</div>
   <div>Can</div>
   <div>Change depending on the iteration of the page</div>
</div>

to retrieve the variable texts with respective to the parent <div id="coolId"> you can use the following solutions:

  • Extracting This using xpath:

    first_child_text = driver.find_element(By.XPATH, "//div[@id='coolId']//following::div[1]").text

  • Extracting Can using xpath:

    second_child_text = driver.find_element(By.XPATH, "//div[@id='coolId']//following::div[2]").text

  • Extracting Change depending on the iteration of the page using xpath:

    third_child_text = driver.find_element(By.XPATH, "//div[@id='coolId']//following::div[3]").text

To extract all the texts from the decendents using xpath:

  all_child_text = driver.find_element(By.XPATH, "//div[@id='coolId']").text
user_74294470
Related