How do I grab a title and link from a website?

user_16547907 0
Viewed 72

I'm currently working on a web scraper on this website (https://www.allabolag.se). I would like to grab the title and link to every result on the page, and I'm currently stuck.

<a data-v-4565614c="" href="/5566435201/grenspecialisten-forvaltning-ab">Grenspecialisten Förvaltning AB</a>

This is an example from the website where I would like to grab the href and >Grenspecialisten Förvaltning AB< as It contains the title and link. How would I go about doing that?

The code I have currently looks like this

import requests
from bs4 import BeautifulSoup

headers = {'User-Agent':  'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1 Safari/605.1.15'}
url = 'https://www.allabolag.se/bransch/bygg-design-inredningsverksamhet/6/_/xv/BYGG-,%20DESIGN-%20&%20INREDNINGSVERKSAMHET/xv/JURIDIK,%20EKONOMI%20&%20KONSULTTJÄNSTER/xl/12/xb/AB/xe/4/xe/3'
r = requests.get (url, headers=headers)
soup = BeautifulSoup(r.text, 'html.parser')

questions = soup.findAll('div', {'class': 'tw-flex'})

for item in questions:
    title = item.find('a', {''}).text
    print(title)

Any help would be greatly appreciated!

Best regards :)

web-scraping python
1 Answers

The results are embedded in the page in Json form. To decode it, you can use next example:

import json
import requests
from bs4 import BeautifulSoup

headers = {
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.1.1 Safari/605.1.15"
}
url = "https://www.allabolag.se/bransch/bygg-design-inredningsverksamhet/6/_/xv/BYGG-,%20DESIGN-%20&%20INREDNINGSVERKSAMHET/xv/JURIDIK,%20EKONOMI%20&%20KONSULTTJÄNSTER/xl/12/xb/AB/xe/4/xe/3"
r = requests.get(url, headers=headers)
soup = BeautifulSoup(r.content, "html.parser")

data = json.loads(
    soup.find(attrs={":search-result-default": True})[":search-result-default"]
)

# uncomment to print all data:
# print(json.dumps(data, indent=4))

for result in data:
    print(
        "{:<50} {}".format(
            result["jurnamn"], "https://www.allabolag.se/" + result["linkTo"]
        )
    )

Prints:

Grenspecialisten Förvaltning AB                    https://www.allabolag.se/5566435201/grenspecialisten-forvaltning-ab
Peab Fastighetsutveckling Syd AB                   https://www.allabolag.se/5566998430/peab-fastighetsutveckling-syd-ab
BayWa r.e. Nordic AB                               https://www.allabolag.se/5569701377/baywa-re-nordic-ab
Kronetorp Park Projekt AB                          https://www.allabolag.se/5567196539/kronetorp-park-projekt-ab
SVENSKA HUSCOMPAGNIET AB                           https://www.allabolag.se/5568155583/svenska-huscompagniet-ab
Byggnadsaktiebolaget Gösta Bengtsson               https://www.allabolag.se/5561081869/byggnadsaktiebolaget-gosta-bengtsson
Tectum Byggnader AB                                https://www.allabolag.se/5562903582/tectum-byggnader-ab
Winthrop Engineering and Contracting AB            https://www.allabolag.se/5592128176/winthrop-engineering-and-contracting-ab
SPI Global Play AB                                 https://www.allabolag.se/5565082897/spi-global-play-ab
Trelleborg Offshore & Construction AB              https://www.allabolag.se/5560557711/trelleborg-offshore-construction-ab
M.J. Eriksson Entreprenad AB                       https://www.allabolag.se/5567043814/mj-eriksson-entreprenad-ab
Solix Group AB                                     https://www.allabolag.se/5569669574/solix-group-ab
Gripen Betongelement AB                            https://www.allabolag.se/5566646427/gripen-betongelement-ab
BLS Construction AB                                https://www.allabolag.se/5569814345/bls-construction-ab
We Construction AB                                 https://www.allabolag.se/5590705116/we-construction-ab
Helsingborgs Fasad & Kakel AB                      https://www.allabolag.se/5567814248/helsingborgs-fasad-kakel-ab
Gat & Kantsten Sverige AB                          https://www.allabolag.se/5566564919/gat-kantsten-sverige-ab
Bjärno Byggsystem AB                               https://www.allabolag.se/5566743190/bjarno-byggsystem-ab
Bosse Sandells Bygg Aktiebolag                     https://www.allabolag.se/5564391158/bosse-sandells-bygg-aktiebolag
Econet Vatten & Miljöteknik AB                     https://www.allabolag.se/5567388953/econet-vatten-miljoteknik-ab
user_100359850
Related