Python Zip list to Dataframe

user_14772220 0
Viewed 959

I'd like zip some lists from html, I use codes like:

html_link = 'https://www.pds.com.ph/index.html%3Fpage_id=3261.html'
html = requests.get(html_link).text
soup = BeautifulSoup(html, 'html.parser')

search = re.compile(r"March.+2021")

for td in soup.find_all('td', text=search):
    link = td.parent.select_one("td > a")

    if link:
        titles = link.text
        links = f"Link : 'https://www.pds.com.ph/{link['href']}"
        dates = td.text
for link, title, date in zip(links, titles, dates):
    dataframe = pd.DataFrame({'col1':title,'col2':link,'col3':date},index=[0])
    print(dataframe)

But the output is not what I expected:

col1 col2 col3
1    P    L    M
  col1 col2 col3
1    D    i    a
...

What I EXPECT is:

Titles Links Dates
...    ...    ...

May I ask if the syntax is correct or what could I do to achieve that?

dataframe python
2 Answers

You can just pass the result from zip directly to pd.DataFrame, specifying the column names in a list:

df = pd.DataFrame(zip(titles, links, dates), columns=['Titles', 'Links', 'Dates'])
user_94737640

If you are trying to create a dataframe from the extracted values then, you need to store them in list before performing zip

from bs4 import BeautifulSoup
import requests
import pandas as pd
import re

html_link = 'https://www.pds.com.ph/index.html%3Fpage_id=3261.html'
html = requests.get(html_link).text
soup = BeautifulSoup(html, 'html.parser')

search = re.compile(r"March.+2021")

titles = []  # to store extracted values in list
links = []
dates = []
for td in soup.find_all('td', text=search):
    link = td.parent.select_one("td > a")

    if link:
        titles.append(link.text)
        links.append(f"Link : 'https://www.pds.com.ph/{link['href']}")
        dates.append(td.text)

dataframe = pd.DataFrame(zip(titles, links, dates), columns=['Titles', 'Links', 'Dates'])
# or you can use
# dataframe = pd.DataFrame({'Titles': titles, 'Links': links, 'Dates': dates})


print(dataframe)

#                                              Titles                                               Links           Dates
# 0  RCBC Lists PHP 17.87257 Billion ASEAN Sustaina...  Link : 'https://www.pds.com.ph/index.html%3Fp=...  March 31, 2021
# 1  Aboitiz Power Corporation Raises 8 Billion Fix...  Link : 'https://www.pds.com.ph/index.html%3Fp=...  March 16, 2021
# 2  Century Properties Group, Inc Returns to PDEx ...  Link : 'https://www.pds.com.ph/index.html%3Fp=...   March 1, 2021
# 3  PDS Group Celebrates 2020’s Top Performers in ...  Link : 'https://www.pds.com.ph/index.html%3Fp=...  March 29, 2021
user_30913980
Related