How to get each element text since he is first sibling of h3 ( all h3 in the page) and without class access [classes change from page to antoher]

Admin 1
Viewed 43

Here is the the text i want to get from scraping, of course i want to get all the first divs that are an h3 first sibling
enter image description here

Here is my code

def get_post_comments(self):
    REQUEST_URL = f'https://mbasic.facebook.com/story.php?story_fbid={self.post_id}&id=415518858611168'
    soup = BeautifulSoup(self.parse_html(REQUEST_URL).content, "html.parser")
    comments = [comment.get('h3') for comment in soup.find_all('h3').next_sibling]

    print(comments)

I'm having multiple errors and i'm newbie

web-scraping selenium beautifulsoup python
1 Answers

You can use CSS selector +:

from bs4 import BeautifulSoup

html_doc = """
<h3>title 1</h3>
<div class="ea">
    This text I want
    <br />
    ...
<div>
<div class="eax">
    ...
<div>

<h3>title 2</h3>
<div class="fa">
    This is also text I want
    <br />
    ...
<div>
<div class="fax">
    ...
<div>
"""

soup = BeautifulSoup(html_doc, "html.parser")

for div in soup.select("h3 + div"):
    print(div.contents[0].strip())

Prints:

This text I want
This is also text I want
user_100359850
Related