Selecting elements at the same node

user_3287147 0
Viewed 119

I am writing Selenium automation scripts in Python. As a process, I need to collect some elements. Below is sample html (I edited to look simple). When I perform a search, I get the search result similar to below.

Now, I need the list of elements in Restaurants and performs some operations. Similarly for Bars, Shopping, Coffee.

The thing is lh and li are at the same node level.

I tried the following xpath but it selects only lh elements

//ul[@id='searchresults']//ancestor::lh

I also gone through helpful Concept XML XLST preceding-sibling and ancestor . But, have not figured out the solution

Sample html code: 

<ul id="searchresults">
    <lh style="">Restaurants</lh>
    <li title="Chamber"><span>Chamber</span></li>
    <li title="Chillies"><span>Chillies</span></li>
    <li title="Sushi Ville"><span>Sushi Ville</span></li>
    <li title="Toasters"><span>Toasters</span></li>
    <li title="Joe Grills"><span>Joe Grills</span></li>
    <lh style="">Bars</lh>
    <li title="Y Cocktails"><span>Y Cocktails</span></li>
    <li title="Z Brewery"><span>Z Brewery</span></li>
    <li title="X Drinks"><span>X Drinks</span></li>
    <lh style="">Shopping</lh>
    <li title="Pacific"><span>Pacific</span></li>
    <li title="Spencers"><span>Spencers</span></li>
    <li title="Hays"><span>Hays</span></li>
    <lh style="">Coffee</lh>
    <li title="Roasters"><span>Roasters</span></li>
    <li title="Coffee Beans"><span>Coffee Beans</span></li>
    <li title="Coffee Cafe"><span>Coffee Cafe</span></li>
</ul>

your help is very much appreciated and please let me know any information I am missing or needed which will help to solve this.

selenium-webdriver selenium xpath css python
2 Answers

You could do the following:

  • iterate over the "list header" elements
  • for each "list header", get the following siblings and collect "tags" (let's name these li element texts as "tags") until the lh sibling is met

Something along these lines:

for lh in driver.find_elements_by_xpath("//ul[@id='searchresults']//lh"):
    restaurant = lh.text

    tags = []
    for element in lh.find_elements_by_xpath("./following-sibling::*"):
        if element.tag_name == 'lh':
            break

        tags.append(element.text)

    print(restaurant, tags)
user_7718480

I figured of something simple by running some javascript in the session that will add a custom attribute with the name of the list header (LH) to each of the following list items (LI), you need to change the custom attribute name to your needs without conflicting with current attributes.

my_js = """let currentLh;
document.querySelectorAll(arguments[0]).forEach(function (elem) {
    if (elem.tagName === "LH") {
        currentLh = elem.textContent.toLowerCase();
    } else {
        elem.setAttribute("my-custom-attr", currentLh);
    }
"""

driver.execute_script(my_js, 'ul#searchresults > *')
my_restaurant_list = driver.find_elemets_by_css_selector('li[my-custom-attr="restaurant"]')
my_bar_list = driver.find_elemets_by_css_selector('li[my-custom-attr="bars"]')
my_shopping_list = driver.find_elemets_by_css_selector('li[my-custom-attr="shopping"]')
my_coffee_list = driver.find_elemets_by_css_selector('li[my-custom-attr="coffee"]')

This uses JS NodeList.forEach function which may not be available in all browsers, if you find an issue there you will have to find a more portable looping solution for all the elements inside the unordered list.

user_35209650
Related