Empty token with Python Requests, but multiple tokens seen in chrome dev tools

user_5392289 0
Viewed 143

I'm trying to use requests to login to a site, navigate to a page, and scrape some data. This question is about the first step (to get in). I cannot fetch the token from the site:

import requests
URL = 'https://coderbyte.com/sl'
with requests.Session() as s:
    response = s.get(URL)
    print([response.cookies])

Result is empty: [<RequestsCookieJar[]>]

This make sense according to the response I'm seeing in Chrome's dev tools. After I login with my username and password, I see four tokens, three of them deleted, but one valid: enter image description here

How can I fetch the valid token?

python-requests python-3.x authentication
1 Answers

you can use the post method to the url you want in order to fetch the token (to pass the login first). For example :

url = "url-goes-here"
url_login = "login-url-goes-here"

with requests.Session() as s:
    # get the link first
    s.get(url)

    payload = json.dumps({
        "email" : "your-email",
        "password" : "your-password"
    })

    headers = {
      'Content-Type': 'application/json'
    }

    response = s.post(url=url_login, data=payload, headers=headers)
    print(response.text)

Based on your question, i assume that if you only use username or password to login, then you can use HTTPBasicAuth() which is provided by requests package.

user_118816540
Related