Selenium find_elements output in non-website based order

user_14896203 0
Viewed 45

Today I opened a ticket on how to find all url links with a specific name in it, which was solved by amazing people here, however, now I'm facing an issue that the output comes in an order that does not correspond to the website.

Current code:

website = 'https://www.abitareco.it/nuove-costruzioni-milano.html'
path = Path().joinpath('util', 'chromedriver')
driver = webdriver.Chrome(path)
driver.get(website)

main = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.NAME, "p1")))

url_list = driver.find_elements(By.XPATH, './/a[contains(@href, "scheda")]')
for x in url_list:
    print(x.get_attribute('href'))

Output:

https://www.abitareco.it/scheda-MILANO2.html
https://www.abitareco.it/scheda-MILANO2-NUOVOCENTROPARCO.html
https://www.abitareco.it/scheda-MARTESANA-GRECO.html
https://www.abitareco.it/scheda-CADORE.html
...

The issue is that the order of the projects on the website is following:

0                   ADRIANO
1                    BAGGIO
2                    BAGGIO
3                     BRERA
4                    CADORE
...

As you can see the first url should be ADRIANO, followed by BAGGIO and BAGGIO2.

How would I work around this issue and fix the sorting?

Thank you!

web-scraping selenium python
1 Answers

I think, issue is with the xpath, if you use this :

//div[@name='p1']/a

and then print the href, I think it would print in web order :

url_list = driver.find_elements(By.XPATH, "//div[@name='p1']/a")
for x in url_list:
    print(x.get_attribute('href'))
user_94440730
Related