Error when scrapin No. with beautifulsoup

user_19733076 0
Viewed 39

I am trying to capture the time of "Rolling" but it doesn't work. Tried different methods, looked up on the in internet but still can't figure out what's wrong. Please help

from bs4 import BeautifulSoup
import requests

url_format = "https://csgoempire.com/"
get_url = requests.get(url_format).text
soup = BeautifulSoup(get_url, 'lxml')
frame = soup.find_all('div', class_ = 'wheel__item wheel__mask absolute w-full h-full top-0 left-0 z-10 wheel__item--visible')

for rezult in frame:
    time = rezult.find('div', class_ = 'text-2xl font-bold font-numeric')
    print(time)

from bs4 import BeautifulSoup
import requests


url_format = "https://csgoempire.com/"
get_url = requests.get(url_format).text
soup = BeautifulSoup(get_url, 'lxml')
frame = soup.find_all('div', class_ = 'wheel__item')

for rezult in frame:
    time = rezult.find('div', class_ = 'text-2xl')
    print(time)
beautifulsoup python
1 Answers

This page you are scraping "https://csgoempire.com/" does not contain any div with that class. So you are not seeing any results. It seems that much of this page is generated via javascript. BS4 does not process javascript.

You will need to implement a 'client' to properly work with javascript. Please see this article, as I think it will have a lot of the building blocks you are looking for: https://pythonprogramming.net/javascript-dynamic-scraping-parsing-beautiful-soup-tutorial/

user_158990350
Related