Selenium, struggle closing a window after checking 100+ boxes across 20 pages, standardized code

user_15113517 0
Viewed 46

On the following website https://icem.data-archive.ac.uk/#step1 For different counties, I will have to select all the HISCO Occupation variables, which means clicking the "more variables" button and checking each box on every page. The number of pages of these HISCO Occupations is variable and depends on the county.

What I've done so far is code in the following format, checking each box and then clicking on the next page, for the maximum number of pages (20 pages):

webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[1]/label/input').click()
webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[2]/label/input').click()
webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[3]/label/input').click()
webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[4]/label/input').click()
webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[5]/label/input').click()
webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[6]/label/input').click()
webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[7]/label/input').click()
webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[8]/label/input').click()
webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[9]/label/input').click()
webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[10]/label/input').click()

webD.find_element_by_xpath('/html/body/div[3]/div/div/div[4]/div/div[1]/ul/li[9]/a').click()

When I try and run this on an example with only 17 pages, the checking each box and flipping pages part works great. My issue lies in clicking the "Apply" button.

In earlier code, only selecting one HISCO Occupation, the following code worked in clicking the Apply button:

WebDriverWait(webD, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[contains(.,'Apply')]"))).click()

This same command doesn't work in this setting. It only works if I have the exact tailored boxes ticked in my code (if I have 17 pages, my code must only span 17 pages), no extra. As I have to do this for each year and each county, I cannot tailor the code and need a general strategy to solve this.

As always, thank you to anyone who helps me!!

IDLE Sends this message:

Traceback (most recent call last):
  File "C:/Users/jessi/Desktop/Selenium/RA_TASK1_ALL_HISCO_CHECKED.py", line 326, in <module>
    webD.find_element_by_xpath('/html/body/div[3]/div/div/div[3]/div[6]/label/input').click()
  File "C:\Users\jessi\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "C:\Users\jessi\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 976, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "C:\Users\jessi\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\jessi\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"/html/body/div[3]/div/div/div[3]/div[6]/label/input"}
  (Session info: chrome=88.0.4324.104)
selenium-chromedriver selenium-webdriver browser-automation selenium python
1 Answers

If you want to select multiple similar elements, try find_elements_by_xpath.

webD.find_elements_by_xpath('/html/body/div[3]/div/div/div//label/input') would return a list all the input elements you want to click regardless of how many of them are listed. Then you can click them all in a for loop.

You may want to use better xpath expressions, like '//div[@class="modal-body ng-scope"]//input[@type="checkbox"]'

So basically you want something like this on each page:

input_elements = webD.find_elements_by_xpath('//div[@class="modal-body ng-scope"]//input[@type="checkbox"]')
for elem in input_elements:
   elem.click()
user_83757830
Related