Extract specific value from a table using Beautiful Soup (Python)

user_12610934 0
Viewed 331

I looked around on Stackoverflow, and most guides seem to be very specific on extracting all data from a table. However, I only need to extract one, and just can't seem to extract that specific value from the table.

Scrape link:

https://gis.vgsi.com/portsmouthnh/Parcel.aspx?pid=38919

I am looking to extract the "Style" value from the table within the link.

Code:

import bs4

styleData=[]

pagedata = requests.get("https://gis.vgsi.com/portsmouthnh/Parcel.aspx?pid=38919") 
cleanpagedata = bs4.BeautifulSoup(pagedata.text, 'html.parser') 

table=cleanbyAddPD.find('div',{'id':'MainContent_ctl01_panView'})
style=table.findall('tr')[3]
style=style.findall('td')[1].text
print(style)
styleData.append(style)
beautifulsoup python
4 Answers

Probably you misused find_all function, try this solution:

style=table.find_all('tr')[3]
style=style.find_all('td')[1].text
print(style)

It will give you the expected output

user_84777380

Could also do something like:

import bs4 
import requests
style_data = []
url = "https://gis.vgsi.com/portsmouthnh/Parcel.aspx?pid=38919"

soup = bs4.BeautifulSoup(requests.get(url).content, 'html.parser')
# select the first `td` tag whose text contains the substring `Style:`.
row = soup.select_one('td:-soup-contains("Style:")')
if row:
    # if that row was found get its sibling which should be that vlue you want
    home_style_tag = row.next_sibling
    style_data.append(home_style_tag.text)

A couple of notes

  • This uses CSS selectors rather than the find methods. See SoupSieve docs for more details.
  • The select_one relies on the fact that the table is always ordered in a certain way, if this is not the case use select and iterate through the results to find the bs4.Tag whose text is exactly 'Style:' then grab its next sibling

Using select:

rows = soup.select('td:-soup-contains("Style:")')
row = [r for r in rows if r.text == 'Style:']
home_style_text = row.text
user_103598500

You can use a CSS Selector:

#MainContent_ctl01_grdCns tr:nth-of-type(4) td:nth-of-type(2)

Which will select the "MainContent_ctl01_grdCns" id, the fourth <tr>, the second <td>.

To use a CSS Selector, use the .select() method instead of find_all(). Or select_one() instead of find().

import requests
from bs4 import BeautifulSoup


URL = "https://gis.vgsi.com/portsmouthnh/Parcel.aspx?pid=38919"

soup = BeautifulSoup(requests.get(URL).content, "html.parser")
print(
    soup.select_one(
        "#MainContent_ctl01_grdCns tr:nth-of-type(4)  td:nth-of-type(2)"
    ).text
)

Output:

Townhouse End
user_123497340

You can use :contains on a td to get the node with innerText "Style" then an adjacent sibling combinator with td type selector to get the adjacent td value.

import bs4, requests

pagedata = requests.get("https://gis.vgsi.com/portsmouthnh/Parcel.aspx?pid=38919") 
cleanpagedata = bs4.BeautifulSoup(pagedata.text, 'html.parser') 
print(cleanpagedata.select_one('td:contains("Style") + td').text)
user_62412350
Related