How can I find a tag in html site that I know it matches a certain pattern in python?

user_15213303 0
Viewed 61

I want to get the width percentage from the winter/spring/summer etc charts in this site: https://www.fragrantica.com/perfume/Christian-Dior/Sauvage-Eau-de-Parfum-48100.html

For example, for the winter season, I want to find this row in the page elements:

<div style="border-radius: 0.2rem; height: 0.3rem; background: rgb(120, 214, 240); width: 90.3491%; opacity: 1;"></div>

I have tried the following

res = requests.get("https://www.fragrantica.com/perfume/Christian-Dior/Sauvage-Eau-de-Parfum- 
                    48100.html", headers={'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) 
                                           AppleWebKit/537.36 (KHTML, like Gecko),
                                           Chrome/88.0.4324.150 Safari/537.36'}
soup = BeautifulSoup(res.content, 'html.parser')
winter_row = soup.select('div[style="rgb(120, 214, 240)"]')
print(winter_row)

I wanted to find that certain html row using the RGB that is unique for each season. The problem is I get an empty list as the output. I want my code to extract the width for each season, day and night from the charts so I know exactly what percentage the votes are.

Do you guys have any idea what should I do?

PS: I also got the name of the perfume from the site and it works, so I know I get a response.

name_row = soup.select('#toptop')[0]
name = name_row.getText().replace('\n', '')
python-requests beautifulsoup css html python
1 Answers

You get an empty list because you use:

soup.select('div[style="rgb(120, 214, 240)"]')

But this expression is looking for an exact match.

But in fact you want to find div elements having a style attribute, and this attribute must contain a background CSS property with a specific value (rgb(120, 214, 240) in your example). So you have to use this CSS attribute selector syntax:

soup.select("[style*='background: rgb(120, 214, 240)']")

Example:

htmlpage = """<!doctype html>
<html lang="en">
  <head>
    <title>Title</title>
  </head>
  <body>
    <div style="border-radius: 0.2rem; height: 0.3rem; background: rgb(120, 214, 240); width: 90.3491%; opacity: 1;"></div>
  </body>
</html>"""

soup = BeautifulSoup(htmlpage, 'html.parser')
extracted = soup.select("[style*='background: rgb(120, 214, 240)']")

print(extracted)

Output:

[<div style="border-radius: 0.2rem; height: 0.3rem; background: rgb(120, 214, 240); width: 90.3491%; opacity: 1;"></div>]

And you should use res.text instead of res.content which gives you the content of the response in bytes, not as text that can be parsed.

user_143945220
Related