Cookie error while trying to authenticate for data scraping

user_10971051 0
Viewed 151

I'm trying to scrape some data from truepush website, but first it needs to be authenticated. So here is what I'm doing:

const loginUrl = 'https://app.truepush.com/api/v1/login'

let loginResult = await axios.get(loginUrl)
    .then(({ headers }, err) => {
        if (err) console.error(err);
        return headers['set-cookie'][0];
    })
    .then((cookie, err) => {
        if (err) console.error(err);
        const splitByXsrfCookieName = cookie.split("XSRF-TOKEN=")[1]
        return splitByXsrfCookieName.split(';')[0];
    }).then(xsrfToken => {
        return axios.post(loginUrl, {
            headers: {
                "Content-Type": "application/json",
                "X-XSRF-TOKEN": xsrfToken
            }
        })
    }).then(res => console.log(res))

It throws xrsfToken on second then response and when I try to login in third response with that xsrf token, it shows me this error:

{
    "status_code": "XSRF-ERROR",
    "status": "ERROR",
    "message": "Cross domain requests are not accepting to this endpoint. If you cleared the cookies, please refresh your browser."
}

I'm not sure what wrong I'm doing :(

node.js web-scraping authentication javascript
1 Answers

The main issue is that the call also requires the original cookie to be sent. You need to keep the original cookie your get from set-cookie header and pass it in cookie header in the second call like cookie: originalCookie. Also in your code, there is no body sent in the POST call.

The following code reproduces the login :

const axios = require("axios");

const originalUrl = 'https://app.truepush.com';
const loginUrl = 'https://app.truepush.com/api/v1/login';

const email = "your-email@xxxxxx";
const password = "your-password";

(async () => {
    await axios.get(originalUrl)
        .then(({ headers }, err) => {
            if (err) console.error(err);
            const cookie = headers['set-cookie'][0];
            return {
                cookie: cookie,
                xsrfToken: cookie.split("XSRF-TOKEN=")[1].split(";")[0]
            };
        })
        .then((data, err) => {
            if (err) console.error(err);
            return axios.post(loginUrl, {
                "email": email,
                "password": password,
                "keepMeLoggedIn": "yes"
            }, {
                headers: {
                    "X-XSRF-TOKEN": data.xsrfToken,
                    "cookie": data.cookie
                }
            })
        })
        .then(res => console.log(res.data))
})();

Output:

{
  status_code: 'SUCCESS',
  status: 'SUCCESS',
  message: 'Login Successful',
  data: {
    id: 'xxxxxxxxxxxxxxxxxxx',
    name: 'xxxxx',
    email: 'xxxxxxx@xxxxxx'
  }
}

Note that both cookie and xsrfToken are consumed by the second promise

user_26143640
Related