BeautifulSoup freeze when using 'class'

user_13654589 0
Viewed 59

Here is my html source code

<h1 class="companyName__9bd88132">Chipotle Mexican Grill Inc</h1>
<div class="description__ce057c5c">Chipotle Mexican Grill, Inc. owns and operates quick serve Mexican restaurants. The Company manages restaurants throughout the United States.</div>

I want to get only the description: Chipotle Mexican Grill, Inc. owns and operates quick serve Mexican restaurants. The Company manages restaurants throughout the United States.

Input:

soup = BeautifulSoup(source_code, "html.parser")
print(soup.find("div", {"class" : "description"})

Output: script hangs

Input:

soup = BeautifulSoup(source_code, "html.parser")
for n in soup.find_all("div", {"class" : "description"}):
    print(n)

Output: Script hangs

Input:

soup = BeautifulSoup(source_code, "html.parser")
for n in soup.find_all("div", {"class" : "description"}):
    print(n)

Output: None

What am I doing wrong?

beautifulsoup html python
2 Answers

{"class" : "description"} will only find tags with class that's exactly equal to description. Your class is only begins with description__....

You can use CSS selector div[class^="description"] to select the required element:

from bs4 import BeautifulSoup


source_code = '''
<h1 class="companyName__9bd88132">Chipotle Mexican Grill Inc</h1>
<div class="description__ce057c5c">Chipotle Mexican Grill, Inc. owns and operates quick serve Mexican restaurants. The Company manages restaurants throughout the United States.</div>'''

soup = BeautifulSoup(source_code, 'html.parser')

print(soup.select_one('div[class^="description"]').text)

Prints:

Chipotle Mexican Grill, Inc. owns and operates quick serve Mexican restaurants. The Company manages restaurants throughout the United States.

Or you can use .find() with lambda:

print(soup.find(class_=lambda t: t.startswith('description')).text)
user_100359850

you said your html looks like this

<h1 class="companyName__9bd88132">Chipotle Mexican Grill Inc</h1>
<div class="description__ce057c5c">Chipotle Mexican Grill, Inc. owns and operates quick serve Mexican restaurants. The Company manages restaurants throughout the United States.</div>

and this is the python which you will use to scrape that html's text

from bs4 import BeautifulSoup

soup = BeautifulSoup(source_code.content, 'html.parser') # or you ca use lxml if you want to
for i in soup.find_all("div", {"class" : "description"}): # or attrs{'class': 'description'} 
    print(i.text)

and you will get an output which is like this

Chipotle Mexican Grill Inc
Chipotle Mexican Grill, Inc. owns and operates quick serve Mexican restaurants. The Company manages restaurants throughout the United States.
user_135368930
Related