Beautifulsoup yields only result

user_13443396 0
Viewed 79

I have searched for a number of similiar problems here, but still confused why my code yields only 1 result, instead of at least 15 on each page.

for pagenumber in range (0,2):
        url = 'https://www.autowereld.nl/volkswagen/?mdl=volkswagen_golf|volkswagen_golf-alltrack|volkswagen_golf-cabriolet|volkswagen_golf-plus|volkswagen_golf-sportsvan|volkswagen_golf-variant&p='
        txt = requests.get(url + str(pagenumber))
        soup = BeautifulSoup(txt.text, 'html.parser')
        soup_table = soup.find('article', class_="item")

        for car in soup_table.findAll('a'):
                link = car.get('href')
                sub_url = 'https://www.autowereld.nl' + link
                print(sub_url)
beautifulsoup python
1 Answers

You are using soup.find to find something with tag "article" and class "item". From the documentation, soup.find only finds one instance of a tag, and you are looking for multiple instances of a tag. Something like

for pagenumber in range (0,2):
    url = 'https://www.autowereld.nl/volkswagen/?mdl=volkswagen_golf|volkswagen_golf-alltrack|volkswagen_golf-cabriolet|volkswagen_golf-plus|volkswagen_golf-sportsvan|volkswagen_golf-variant&p='
    txt = requests.get(url + str(pagenumber))
    soup = BeautifulSoup(txt.text, 'html.parser')
    soup_articles = soup.findAll('article', class_="item")

    for article in soup_articles
        for car in article.findAll('a'):
            link = car.get('href')
            sub_url = 'https://www.autowereld.nl' + link
            print(sub_url)

might work. I also recommend using find_all instead of findAll if you're using bs4 since the mixed case versions of these methods are deprecated, but that's up to you.

user_108648240
Related