Python3 Beautifulsoup4 extracting text from multiple container siblings

user_14166428 0
Viewed 42

I am new to python, and I am trying to using beautifulsoup to extract only the text from a group of tags. The first tag is 'name' the second is 'date' I can grab the text either from name, or date just not together. Here is the html code to the page I am trying to scrape

<div class="results">
 <h1>
   Info Records
 </h1>
 <div class="group">
  <a class="name" href="https://" target="_blank">
   Firstname, Lastname
  </a>
  <br/>
  <span class="date">
   8/24/2020: Text info
  </span>
 </div>
 <div class="group">
  <a class="name" href="https://" target="_blank">
   Different Firstname, Different Lastname
  </a>
  <br/>
  <span class="date">
   8/23/2020: Different Text Info
  </span>
 </div>

for name I use this code which pulls the names, and prints them to terminal for the dates I change the class name to 'date'

for arrest in soup.find_all('a', {'class': 'name'}):
    name = arrest.text
    print(name)

The html has about 20 names, and dates I only posted the first 2. When I try to print both classes together it doesn't work.

test = soup.find_all("div",  {"class": ["name", "date"]})
print(test)

Also what is working doesn't write to text file. Ideally what I am trying to accomplish is something like this to be added to a output file:

firsname lastname
8/24/2020 Text info
firstname last name
8/23/20920 different text info

Any advice would be helpful. I've been reading all day trying to figure it out.

beautifulsoup python-3.x
1 Answers

Option 1: use a CSS selector.

Option 2: use zip()

1:

from bs4 import BeautifulSoup

html = """YOUR ABOVE HTML SNIPPET"""

soup = BeautifulSoup(html, "html.parser")

with open("output.txt", "w") as f:
    # select `name` and `date` class
    for tags in soup.select(".name, .date"):
        f.write(tags.text.strip() + "\n")

2:

with open("output.txt", "w") as f:

    for name, date in zip(
        soup.find_all("a", {"class": "name"}), soup.find_all("span", {"class": "date"})
    ):
        f.write(name.text.strip() + "\n")
        f.write(date.text.strip() + "\n")

output.txt:

Firstname, Lastname
8/24/2020: Text info
Different Firstname, Different Lastname
8/23/2020: Different Text Info
user_123497340
Related