Extract about into from hipages

user_13408417 0
Viewed 51

I want to extract this info from https://hipages.com.au/connect/tigerelectricalservices

enter image description here

This is what my code looks like:

description = []
    try:
        description.append(
            soup.select('div', class_="sc-bdnxRM col__Col-sc-15n4ng3-0 icIRSI itbbRL").text)
        print(f"Description: {description}")
    except:
        description.append(" ")

But this does not work.

Can someone help me with this?

Thanks :)

web-scraping beautifulsoup css html python
1 Answers

Desired tag's class name gets changed dynamically with js, and ends up being <div color="tint6" class="sc-dlnjwi btKpNj"> (check website page source). So grabbing that tag will give us desired output:

from bs4 import BeautifulSoup
import requests

res = requests.get('https://hipages.com.au/connect/tigerelectricalservices')

soup = BeautifulSoup(res.text, 'lxml')
parent = soup.find('div', class_='sc-dlnjwi btKpNj').select('div> div:nth-child(2)')

text = parent[0].get_text()

Note that select returns list containing given selectors, so we need to use indexing to get the actual item. Output would look like:

' Welcome to Tiger Electrical Services  Our electricians have extensive experience in the Electrical industry offering a wide range of services. With over 12 years of experience we can offer a wide range of services and understand that our customers expect honesty, integrity and reliability.\r Our business was initially established to provide electrical solutions to the outer suburbs of Berwick, Narre Warren, Beaconsfield, Pakenham and Cranbourne.\r .... '
user_119235580
Related