Beautiful Soap find / try different class elements with possible 'NoneType' errors

Viewed 199

I'm using beautiful soup to get elements off a url. There are elements on the page that the class name changes depending on the value and percentage (up, down, flat). Obviously, if I try soup.find('div', class_='down-indicator') and it is a positive indicator I will get an error like this: AttributeError: 'NoneType' object has no attribute 'find'. I need a pythonic way to loop through or try the 3 possible class options without the error breaking my code.

I've also tried res = soup.find('div', class_='down-indicator') to then do a if res is None... but it does not return None it returns an error.

Any info and learning would be greatly appreciated! Thank you in advance and this is my terrible attempt at some code:

These 2 urls have different class names for same area depending on on element.

url = 'http://www.stockx.com/adidas-yeezy-boost-380-lmnte-infants'
url = 'http://www.stockx.com/adidas-yeezy-boost-350-oxford-tan'

And these are the exact find() functions for the elements because it can be up, down, or flat:

soup.find('div', class_='last-sale-value market-dir market-dir-down').find('div', class_='dollar').text
soup.find('div', class_='last-sale-value market-dir market-dir-up').find('div', class_='dollar').text
soup.find('div', class_='last-sale-value market-dir market-dir-static').find('div', class_='dollar').text


driver = webdriver.Firefox()
driver.get(url)
content = driver.page_source
soup = BeautifulSoup(content, 'lxml')

try:
    result = soup.find('div', class_='up-indicator').text
    result = soup.find('div', class_='down-indicator').text
    result = soup.find('div', class_='flat-indicator').text
except:
    pass
1 Answers

In both cases, the solution is fairly simple:

soup.select_one('div.dollar').text

The output is the target dollar amount of the relevant page.

Related