How to transfer bs4.element.ResultSet to date/string?

user_14227989 0
Viewed 158

I want to extract date and summary of an article in a website, here is my code

from bs4 import BeautifulSoup
from selenium import webdriver
full_url = 'https://www.wsj.com/articles/readers-favorite-summer-recipes-11599238648?mod=searchresults&page=1&pos=20'
url0 = full_url
browser0 = webdriver.Chrome('C:/Users/liuzh/Downloads/chromedriver_win32/chromedriver')
browser0.get(url0)


html0 = browser0.page_source
page_soup = BeautifulSoup(html0, 'html5lib')
date = page_soup.find_all("time", class_="timestamp article__timestamp flexbox__flex--1")
sub_head = page_soup.find_all("h2", class_="sub-head")
print(date)
print(sub_head)

I got the following result, how can I obtain the standard form ?(e.g. Sept. 4, 2020 12:57 pm ET; This Labor Day weekend, we’re...)

[<time class="timestamp article__timestamp flexbox__flex--1">
        Sept. 4, 2020 12:57 pm ET
      </time>]
[<h2 class="sub-head" itemprop="description">This Labor Day weekend, we’re savoring the last of summer with a collection of seasonal recipes shared by Wall Street Journal readers. Each one comes with a story about what this food means to a family and why they return to it each year.</h2>]

Thanks.

web-scraping beautifulsoup python
1 Answers

Try something like:

for d in date:
   print(d.text.strip())

Given your sample html, output should be:

Sept. 4, 2020 12:57 pm ET
user_94480900
Related