Coding return NULL

Nguyen Huu Phuc 0
Viewed 21
import requests # request img from web
from bs4 import BeautifulSoup

imglinks = []

for i in range(1, 26):
    resp = requests.get('https://******/************.html')
    soup = BeautifulSoup(resp.content, "html.parser")
    links = soup.select('.container > .****** > .img')
    for link in links:
        imglinks.append(link['src'])
    
print(links)

My code return NULL (actually it return []) maybe there are some problem with my soup.select ?

beautifulsoup python-requests
1 Answers

Sometimes, BeautifulSoup is unable to find select elements within a webpage. You might either need to render the page using javascript by using the html_session library or you have been blocked by the website for trying to access it using a bot.

If it isn't either of those reasons, i suggest manually scraping it off by finding indexes in request.text

walker0
Related