beautifulsoup - scraping an item through class with condition

user_10467473 0
Viewed 53

for example I have this html

<div class="item-1">a</div>
<div class="item-3">b</div>
<div class="item-6">c</div>
<div class="item-8">aaaaaa</div>
...... item-x keep increasing randomly on it class
<div class="item-100">aaaaaa</div>

I want to scrap all of the class item-X where the value of X is between 5 to 10

I know how to search with a partial class name

text = soup.select('div[class*="item-"]')

but I don't know how to add conditions for it

beautifulsoup html
2 Answers

You can simply use for loop.

import bs4 as bs

html = """
<div class="item-1">a</div>
<div class="item-3">b</div>
<div class="item-6">c</div>
<div class="item-8">aaaaaa</div>
<div class="item-100">aaaaaa</div>
"""

soup = bs.BeautifulSoup(html, 'lxml')

for i in range(5, 10):
    text = soup.select('div[class*="item-' + str(i) + '"]')
    if text:
        print(text)
user_193255780

You can use multiple CSS selectors joined by ,:

html_doc = """\
<div class="item-1">a</div>
<div class="item-3">b</div>
<div class="item-6">c</div>
<div class="item-8">aaaaaa</div>
<div class="item-100">aaaaaa</div>
"""

soup = BeautifulSoup(html_doc, "html.parser")

texts = soup.select(",".join(f"div.item-{i}" for i in range(5, 11)))
for text in texts:
    print(text)

Prints:

<div class="item-6">c</div>
<div class="item-8">aaaaaa</div>
user_100359850
Related