How to get html page with requests for parsing?

Viewed 48

I am new in parsing of websites and I have problem to parse site. When I send request

import requests
from fake_useragent import UserAgent

ua = UserAgent()

headers = {
    "User-Agent": str(ua.random)
}


def get_html(url):
    session = requests.session()
    request = session.get(url, headers=headers)
    if request.ok:
        html = request.text
        session.close()
        return html
    session.close()

site return error 403. I found out that there is a protection against parsing, and you need to send cookie, but sample

cookie = {"__cf_bm"="AmFUHZm_M8VuNyER6T1hrgzR8iAy2YXugwG1rn.vCpU-1662910877-0-ATRIGhsUZNyoD6wztO1JvHCyAdzv5r7zEitRZWrG3051TQQhGnONmP6bHDoil1XrUqZcmkPDcWImr2I3WU3nleI="}
request = session.get(url, headers=headers, cookies=cookie)

which I took from the header Set-cookie did not lead to a result. From browser I can receive this cookie and it will work

headers = {
    "User-Agent": str(ua.random),
    "Cookie": "__Secure-access-token=3.0.2MYFHfKnT-GBYqUr9ARDsw.85.l8cMBQAAAABjCebGOQ7IqaN3ZWKgAICQoA..20220911164138"
              ".xkQl4cZUk82fBXlXw6X7m6u7ZR2RZ0i_XA1I25bZSEU; "
              "__Secure-refresh-token=3.0.2MYFHfKnT-GBYqUr9ARDsw.85.l8cMBQAAAABjCebGOQ7IqaN3ZWKgAICQoA"
              "..20220911164138.xxhYmm0vHYtq2sp0DV2njRInUTVktm6cI9sXjBd8xHk; __Secure-ab-group=85; "
              "__Secure-user-id=0; __cf_bm=5DrABSw5lqarljTnDX3YXCjTMsbW2oZOsy2vZjKnLfA-1662907298-0-AeNr"
              "+1ca4CeHQagStyYew4aWBL5hhIq03ugWx0nrIWL75fwcV9exyxLgMDRge0T52fQT"
              "/XQFeYdo489nANOhQ1GXOPKIHuX1SYI0yH21RC08QTyU1OFD6zzzAwBJUEPamUzpP+DThr1vo6y9eydSFplal56KMw"
              "/MpDltBUmAKrA4 "
}

I tried to parse https://seller.ozon.ru/news. Please, help to solve problem and correctly send the request and take the correct cookies format

0 Answers
Related