I keep getting too many links from scraped website

user_14719410 0
Viewed 37

I'm scraping information from the table on the given website. It returns me the length of 184 even though there are only 180 links in the table

from bs4 import BeautifulSoup //importing stuff
from urllib.request import Request, urlopen

site = "http://www.voltwo.webd.pl/1-indexy/index-5-opracowania/01-maturalne-KINEMATYKA.html"
hdr = {'User-Agent': 'Mozilla/5.0'}
req = Request(site, headers=hdr)
page = urlopen(req)
soup = BeautifulSoup(page, features="html5lib")
table = soup.find(id='table1') //id of the table in website

links = []

for link in table.findAll('a'):
    links.append(link.get('href')) //the length of links should be 180, and it's 184
beautifulsoup python
1 Answers

Seems like the urls you're after are all pdf files but you also get index.php as one of the results.

So, just filter out those anchors that you want and there you have your 180 results.

Here's how:

import requests
from bs4 import BeautifulSoup

url = "http://www.voltwo.webd.pl/1-indexy/index-5-opracowania/01-maturalne-KINEMATYKA.html"
page = requests.get(url).text
soup = BeautifulSoup(page, "html.parser").find_all("td")

anchors = [a.find("a")["href"] for a in soup if a.find("a") and "pdf" in a.find("a")["href"]]
print(len(anchors))

Output: 180

user_61067910
Related