BS4 getting value from different div with the same class

user_13203366 0
Viewed 175

I'm trying to scrape a site that has two types of prices, normal price, and discounted prices.

Normal Price HTML:

<p class="plp__price__325EX">
   <span aria-label="$29.99">$29.99</span>
</p>

Discounted price HTML:

<p class="plp__price__325EX plp__salePrice__2ExZ2 plp__price__325EX">
       $11.99 // trying to get this price
       <span class="plp__priceStrikeThrough__2MAlQ plp__price__325EX">$15.99</span>
    </p>

Normal price code:

try:
    normal_price_element = item.find('p', {'class', 'plp__price__325EX'})
    normal_price = normal_price_element.find('span').text
except:
    normal_price = ''

Discounted price code:

try:
    disc_price = item.find('p', {'class', 'plp__salePrice__2ExZ2'}).text
except:
    disc_price = ''

The only difference between these code snippets is the class name. The problem is that the span in the p class contains plp__price__325EX so the normal price code will run when it shouldn't and get the normal price instead of the discounted price.

beautifulsoup python
1 Answers

Assuming you have this document:

from bs4 import BeautifulSoup


html_doc = """
<p class="plp__price__325EX plp__salePrice__2ExZ2 plp__price__325EX">
       $11.99
       <span class="plp__priceStrikeThrough__2MAlQ plp__price__325EX">$15.99</span>
</p>

<p class="plp__price__325EX">
   <span aria-label="$29.99">$29.99</span>
</p>"""


soup = BeautifulSoup(html_doc, "html.parser")

Then you can use lambda function to locate <p> tag that contains only one class plp__price__325EX:

# find <p> tag that contains only *ONE* class="plp__price__325EX"
normal_price = soup.find(
    lambda tag: tag.name == "p"
    and tag.get("class", []) == ["plp__price__325EX"]
)

dis_price = soup.find("p", class_="plp__salePrice__2ExZ2")


print(normal_price.get_text(strip=True))
print(dis_price.contents[0].strip())

Prints:

$29.99
$11.99

Or you can use CSS selectors:

normal_price = soup.select_one(
    "p.plp__price__325EX:has(>[aria-label])"
).text.strip()
dis_price = soup.select_one("p.plp__salePrice__2ExZ2").text.split()[0]

print(normal_price)
print(dis_price)

Prints:

$29.99
$11.99
user_100359850
Related