The result of parsing petshop.ru is not displayed

user_18899084 0
Viewed 31

I'm trying to parse the page https://www.petshop.ru/catalog/cats/veterinary_feed/dlya_koshek_pri_zapore_fibre_response_fr31_5789/, but it doesn't work.

    import requests
from bs4 import BeautifulSoup as BS

r = requests.get ("https://www.petshop.ru/catalog/cats/veterinary_feed/dlya_koshek_pri_zapore_fibre_response_fr31_5789/")
html = BS (r.content, 'html.parser')

for el in html.select (".style_product_head__ufClP > .style_tablet__bK5he style_desktop__3Zkvu"):
title = el.select ('.style_list__3V0_P > .style_price_wrapper__1HT8P')
print ( title[0].text )

I do according to the model, because unfamiliar with python:

import requests
from bs4 import BeautifulSoup as BS

r = requests.get ("https://stopgame.ru/review/new/izumitelno/p1")
html = BS (r.content, 'html.parser')

for el in html.select (".items > .article-summary "):
title = el.select ('.caption > a')
print ( title[0].text )

I expect to see the following result: Обычная цена

Ideally, it would also be interesting to know how to display a result of this kind: petshop.ru: Обычная цена 3 125 ₽ , because I plan to implement the parsing of several more sites to track prices for this cat food :)

python-requests beautifulsoup parsing python
1 Answers

All the data you want is in the HTML source and it comes as a JSON object in a <script> tag. You can easily target that and parse it.

Here's how:

import json
import re

import requests

url = "https://www.petshop.ru/catalog/cats/veterinary_feed/dlya_koshek_pri_zapore_fibre_response_fr31_5789/"

headers = {
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:99.0) Gecko/20100101 Firefox/99.0",
}

data = (
    json.loads(
        re.search(
            r'"onProductView",\s(.*)\);',
            requests.get(url, headers=headers).text
        ).group(1)
    )
)

print(f"{data['product']['name']} - {data['product']['price']} руб.")

Output:

Для кошек при запоре - 3125 руб.
user_61067910
Related