Python requests to access OAUTH website content - SNL Finance

Question

I've been banging my head up against the wall trying to retrieve content from the news source "SNL Finance". I have valid credentials, so in theory I should be able to programmatically access their news content.

In short, I've tried executing the below script but with no success:

s = requests.Session()

client_id = "..."
client_secret = "..."
token_url = "https://www.snl.com/SNL.Services.Security.Service/oauth/token"
protected_url = "https://www.snl.com/web/client?auth=inherit#news/article?id=40666532&KeyProductLinkType=14"

request_data = {
    "client_id": client_id,
    "client_secret": client_secret,
    "scope": "https://www.snl.com",
    "grant_type": "refresh_token",
    "refresh_token": refresh_token
}

token_response = s.post(token_url, data=request_data)
### token response is in jwt format, including token_type, expires_in, scope, etc. ###
token = json.loads(token_response.text)["access_token"].split('>')[1].split('<')[0]
request_data["token"] = token

article = s.post(protected_url, headers=request_data)

Sadly, this fails. I end up with a 200 response, but it appears to just be the login page (honestly not entirely sure what I'm looking at).

For more background, I've included browser information as it populates throughout the authentication process:

Attempt to visit protected url, redirected to the below url (omitted snl base):

/web/client?auth=inherit&contextType=external&username=string&enablePersistentLogin=true&OverrideRetryLimit=0&SwitchGetToPostLimit=50000&contextValue=%2Foam&password=secure_string&challenge_url=https%3A%2F%2Fwww.snl.com%2Fweb%2Fclient%3Fauth%3Dinherit&request_id=-6149669210818920852&authn_try_count=0&locale=en_US&resource_url=https%253A%252F%252Fwww.snl.com%252FInteractiveX%252FDefault.aspx%253Ftarget%253Dnews%25252Farticle%25253Fid%25253D40666532%252526KeyProductLinkType%25253D14%2526SNL3%253D1

Request headers are shown here.

Upon entering login / password, a token is retrieved and the protected page loads.

Request cookies are as shown here.

Also, I'm a bit confused at to why the token value of SNL_OAUTH_TOKEN in the above link (second link) differs from what is shown in the jwt token response I receive from my script.

Any guidance here will be hugely appreciated. I'm also happy to send any other non-personal information that proves useful.

Thank you!

Python requests to access OAUTH website content - SNL Finance

1 Answers