webscrape jobserve popup with selenium

Question

I am trying to apply for jobs on jobserve using web scraping. After clicking apply on the web page, Selenium cannot locate any of the elements of the pop up:

driver.get("https://www.jobserve.com/gb/en/JobSearch.aspx?shid=415E7EF3D52E66613550")

jobs = driver.find_elements_by_class_name("jobsum")
# jobs[1].click()

data = driver.find_element_by_id("td_jobpositionlink")
# print(data.text)
if "Data Scientist" in data.text:
    driver.find_element_by_link_text("Apply").click()
    time.sleep(5)
    fills = driver.find_element_by_name("rptAppMand$ctl00$Ques$Q0006_ans")
    print(fills.text)

I keep getting this error.

Traceback (most recent call last):
  File "/Users/mac/PycharmProjects/pythonProject/jobserve.py", line 30, in 
    fills = driver.find_element_by_xpath('//*[@id="JobApplyScroll_outercontainer"]/div/div[1]/div')
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 394, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 976, in find_element
    return self.execute(Command.FIND_ELEMENT, {
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[@id="JobApplyScroll_outercontainer"]/div/div[1]/div"}
  (Session info: chrome=90.0.4430.212)


Process finished with exit code 1

How can I fill this form?

user_12730112 · Answer

There is the iframe. You need to switch to it.

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC


driver = webdriver.Chrome(executable_path='/snap/bin/chromium.chromedriver')

driver.get('https://www.jobserve.com/gb/en/JobSearch.aspx?shid=415E7EF3D52E66613550')
wait = WebDriverWait(driver, 20)
jobs = driver.find_elements_by_css_selector(".jobsum")
data = driver.find_element_by_id("td_jobpositionlink")
print(data.text)
if "Data Scientist" in data.text:
    driver.find_element_by_link_text("Apply").click()
    wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "#appFrame")))
    email = driver.find_element_by_css_selector(".questionblock2>.questionInput>input")
    email.send_keys("test")
    #  continue filling in fields

driver.close()
driver.quit()

And the input is done:

webscrape jobserve popup with selenium

1 Answers