Setting a default for nosuchelementexception for multiple variables in python

user_7739895 0
Viewed 36

So I am scrapping multiple rows of a table and many of them are either available or not for different pages. What I want to do is to detect which field is not available and supply it on a variable and set a default variable i.e., None to it. For eg,

try:
        field1 = driver.find_element(By.XPATH, value="somexpath")
        field2 = driver.find_element(By.XPATH, value="somexpath")
        field3 = driver.find_element(By.XPATH, value="somexpath")
        field4 = driver.find_element(By.XPATH, value="somexpath")
        field5 = driver.find_element(By.XPATH, value="somexpath")

        dict_ = {"field1":field1, "field2": field2.....}

except NoSuchElementException:
       # some code to detect which element not found and supply a default value None to it
       defaultVaule = None

Please help.

defaultdict web-scraping selenium error-handling python
3 Answers

You can inject if else None statement as follows:

field1 = driver.find_element(By.XPATH, value="somexpath")
field1=field1.text if field1 else None
user_128484110

You may want to reverse the order - initialize using default values and populate if retrieval succeeds element-wise.

dict_ = {"field1":None, "field2": None.....}

try:
        dict_["field1"] = driver.find_element(By.XPATH, value="somexpath")
try:
        dict_["field2"] = driver.find_element(By.XPATH, value="somexpath")
user_7100300

I came up with a better way the same day I asked question to handle exception through a separate function. It just I forget to update it.

first import

from selenium.common.exceptions import NoSuchElementException

then call this function

def box_items():
    try:
        ele = driver.find_element(By.XPATH, value=//*[@id='some_id']/div/a).text
        except NoSuchElementException:
            ele = "None"
        return ele
user_77398950
Related