Beutifulsoup sometimes print None sometimes

user_13564172 0
Viewed 48

I tried to scrape a image from a reddit post. But when I run this code snippet It show me html snippet sometimes, but sometimes it prints None (NO Error occurred). Anybody can tell me why? Here is the code.

from bs4 import BeautifulSoup
import requests

source = requests.get('https://www.reddit.com/r/programmingmemes/').text
soup = BeautifulSoup(source, 'lxml')

img = soup.find('div', class_='_3Oa0THmZ3f5iZXAQ0hBJ0k')
print(img)
python-requests beautifulsoup python
1 Answers

Check the return code of the request:

from bs4 import BeautifulSoup
import requests

source = requests.get('https://www.reddit.com/r/programmingmemes/')

if source.status_code == 200:
    soup = BeautifulSoup(source.text, 'lxml')

    img = soup.find('div', class_='_3Oa0THmZ3f5iZXAQ0hBJ0k')
    print(img)
else:
    print(f"Error (code {source})")

Also check if the class is constant during time (it may be randomized).

user_149926640
Related