Issue Scraping with Python and BeautifulSoup

user_6928447 0
Viewed 37

I've been scraping this website for a year now, however they recently changed the layout of the website and for some reason I can't make it work anymore. I'm using Python and BeautifulSoup.

I'm basically trying to get the data in the tables from this link: https://www.loto.ro/?p=3872

This is the code that used to work with the old layout and I adapted it for the current layout of the website:

            website_result = requests.get("https://www.loto.ro/?p=3872")
            src = website_result.content
            soup = BeautifulSoup(src, 'lxml')

            for i in range(0, 8):
                table_title = soup.select(".content .content-info .rezultate-extrageri-content.resultDiv .button-open-details")[i].get_text().strip()

                if "6/49" in table_title:
                    images = soup.select(".content-info .rezultate-extrageri-content.resultDiv "
                                         ".info-rezultat .numere-extrase img[src]")

                    if len(images) > 0:
                        table = soup.select(".content .content-info .rezultate-extrageri-content.resultDiv .results-table")[i]

In debug mode, my code gets stuck at the "table_title" line without giving me any error or traceback so I don't even know what the problem is.

Any ideas? Thanks.

beautifulsoup python
1 Answers

The URL the results are coming form is indeed new, as it has "newLottoSite" in it.

Try this:

import pandas as pd
import requests
from tabulate import tabulate

new_url = "https://www.loto.ro/loto-new/newLotoSiteNexioFinalVersion/web/app2.php/jocuri/649_si_noroc/rezultate_extragere.html"

headers = {
    "user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.41 Safari/537.36",
    "referer": "https://www.loto.ro/?p=3872",
}

df = pd.read_html(requests.get(new_url, headers=headers).text, flavor="lxml")[0]
print(tabulate(df, headers="keys", tablefmt="psql"))

This should output:

+----+---------------------------------------+---------------------------------------+---------------------------------------+---------------------------------------+
|    | CAT.                                  | Numar castiguri                       | Valoare castig                        | Report                                |
|----+---------------------------------------+---------------------------------------+---------------------------------------+---------------------------------------|
|  0 | I (6/6)                               | REPORT                                | 272.80920                             | 4.289.31280                           |
|  1 | II (5/6)                              | 5                                     | 18.18728                              | -                                     |
|  2 | III (4/6)                             | 285                                   | 31907                                 | -                                     |
|  3 | IV (3/6)                              | 4.563                                 | 3000                                  | -                                     |
|  4 | Fond total de castiguri: 4.608.075,60 | Fond total de castiguri: 4.608.075,60 | Fond total de castiguri: 4.608.075,60 | Fond total de castiguri: 4.608.075,60 |
+----+---------------------------------------+---------------------------------------+---------------------------------------+---------------------------------------+
user_61067910
Related