How can I get the next child in BeautifulSoup

user_11072773 0
Viewed 64

This is the HTML and code I have:

<a class="card__article-link" href="linktoarticle" title="articletitle">
<span class="card__egida">TEXT</span>
<span class="card__title ">TITLE</span>
<span class="card__subtitle">SUBTITLE</span>
</a>

import requests
from bs4 import BeautifulSoup
r = requests.get("link").text
soup = BeautifulSoup(r, "html.parser")

for span in soup.find_all("span", {"class": "card__egida"}):
    print(span.get_text())

The code correctly prints TEXT but I want the code to also print TITLE and SUBTITLE. I have tried with nextSibling but with no success. How can I do that?

python-requests web-scraping beautifulsoup python-3.x python
1 Answers

You can use .find_next() to get next elements:

from bs4 import BeautifulSoup


txt = '''<a class="card__article-link" href="linktoarticle" title="articletitle">
<span class="card__egida">TEXT</span>
<span class="card__title ">TITLE</span>
<span class="card__subtitle">SUBTITLE</span>
</a>'''

soup = BeautifulSoup(txt, 'html.parser')    

for span in soup.find_all("span", {"class": "card__egida"}):
    egida = span.get_text()
    title = span.find_next(class_='card__title').get_text()
    subtitle = span.find_next(class_='card__subtitle').get_text()

    print(egida)
    print(title)
    print(subtitle)

Prints:

TEXT
TITLE
SUBTITLE

Or: you can select the parent <a> and then search for title, subtitle, etc...:

for a in soup.select('a.card__article-link'):
    egida = a.select_one('.card__egida').get_text()
    title = a.select_one('.card__title').get_text()
    subtitle = a.select_one('.card__subtitle').get_text()
user_100359850
Related