I am trying to get the information from here
https://www.philips.com/a-w/security/security-advisories
I want each category article to be assigned to a dataframe
So for example First article-->first row of the datafrane, second article -->second row of the datafrane..etc
For the beginning I am trying the following code:
import pandas as pd
import requests
from bs4 import BeautifulSoup
from requests.models import ContentDecodingError
url = "https://www.philips.com/a-w/security/security-advisories"
# get the html content of the url
html_content = requests.get(url).text
# parse the html content
soup = BeautifulSoup(html_content, 'html.parser')
span_content = []
for span in soup.find_all("span", class_="p-body-copy-02"):
span_content.append(span.text)
The span_content contains information in the following form where each new category is starting after the publication and update date fields:
['Publication Date:\xa02022 August 25',
'Update Date: 2022 August 25',
'Philips is currently monitoring... specific to their Philips’ products.',
Publication Date:\xa02022 August 18',
'Update Date: 2022 August 18',
'Philips is currently monitoring...specific to their Philips’ products.',
etc]
I am trying the following code to get rid of the publication date and update date:
def delete_date(span_content):
for i in range(len(span_content)):
if span_content[i] == 'Publication Date:' or span_content[i] == 'Update Date:':
span_content.pop(i)
break
return span_content
delete_date(span_content)
However this is working.
So how do I get rid of the publication date and update date and cast the information into a dataset?
index Info
0 Philips is currently monitoring... specific to their Philips’ products
1 Philips is currently monitoring...specific to their Philips’ products
... ...
etc etc