How to access a table in HTML without any name with Beautifulsoup

user_14385208 0
Viewed 112

I am trying to scrape a table from url. The table seems to have no name. I have scraped the links and its text to csv using below code.

from bs4 import BeautifulSoup
import requests
import re

url = 'https://www.sbp.org.pk/smefd/circulars/2020/index.htm'

r = requests.get(url)

soup = BeautifulSoup(r.content, 'html.parser')


anchor = soup.find_all('a',href = re.compile('2020'))


file_name = "Circulars.csv"
f = open(file_name,'w')
header = 'Cir_Name ,Links\n'
f.write(header)

for link in anchor:
    href = link.get('href')
    text = link.getText()
    text1 = text.replace("\n", "")
    f.write(text1.replace(',','|') + "," + href.replace("," ,"|") +"\n")

print("Done")

What I need is to scrape the table as it is. currently I am able to get just links. I need other columns too.

I have tried following code but failed. I am able to get to the table but thats not enough

from bs4 import BeautifulSoup
import requests
import re

url = 'https://www.sbp.org.pk/smefd/circulars/2020/index.htm'

r = requests.get(url)

soup = BeautifulSoup(r.content, 'html.parser')


table = soup.find_all('table',attrs={"width": "95%",'border':'0','cellpadding':"1"})



tablee = soup.find_all('tr',attrs={'table',attrs={"width": "95%",'border':'0','cellpadding':"1"}})



print(tablee)
beautifulsoup html python
3 Answers

To save data to CSV, you can use this example:

import requests
import pandas as pd
from bs4 import BeautifulSoup


url = 'https://www.sbp.org.pk/smefd/circulars/2020/index.htm'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')

table = soup.select_one('table[width="95%"]:not(:has(table))')

all_data = []
for row in table.select('tr:not(:has(td[colspan]))'):
    tds = [td.get_text(strip=True).replace('\n', ' ').replace('\t', ' ') for td in row.select('td') if td.get_text(strip=True)]
    tds += [row.find_previous('td', {'colspan': '4'}).get_text(strip=True).replace('\n', ' '), row.a['href']]
    all_data.append(tds)

df = pd.DataFrame(all_data)
print(df)
df.to_csv('data.csv', index=False, header=False)

Saves data.csv (screenshot from LibreOffice):

enter image description here

user_100359850

Since there is only one table in the page, it is not necessary to provide the attributes. U can simply do

table = soup.find('table')

in order to get the table. Then, u can do whatever u want with the table.

Hope that this helps!

user_123098080

use lambda. Gives you the table + rows.

table = bs.find(lambda tag: tag.name=='table') 
rows = table.findAll(lambda tag: tag.name=='tr')
user_63641000
Related