The same request to external API works in go, postman, curl, but doesn't work in node.js

user_9560929 0
Viewed 359

I have a strange problem while performing request to external API. Basically I got 403 error response for the HTTP request, but... only when calling it from node environment. I tried with curl through terminal and it's OK. I also tried from Postman and native go with net/http package and everything is just fine, I got 200 response. But no matter what tool I use in node.js (request, axios, unirest, got) I can't make it. I would love to share entire request with you, but unfortunately I can't :(

All I can show you is some response details, when I call it from node.js environment.

These are headers sent by the server when I got 403:

  headers: {
    server: 'AkamaiNetStorage',
    'content-length': '228',
    'content-type': 'application/json',
    'x-edge-error': 'halt',
    'cache-control': 'max-age=122',
    date: 'Thu, 21 Oct 2021 16:17:41 GMT',
    connection: 'close'
  },

And the part of response data looks like this:

  data: {
    edge_error: 'halt',
    ref_id: '18.9d6656b8.1634833061.4b9f14b',
  }

I am aware that question is strange and you probably won't be able to help me with not so many details, but I will try anyway. So If anyone have idea what might help, feel free to propose an answer.

Best regards ;)

node.js request api http javascript
1 Answers

This is likely due to TLS fingerprinting. Akamai is able to detect that you're using Node.JS based on information gathered during the TLS handshake (cipher suites, etc.) when you send a request. The website must have set a rule through Akamai to block Node.

Try something like this:

const tls = require('tls');
const https = require('https');

const defaultCiphers = tls.DEFAULT_CIPHERS.split(':');
const shuffledCiphers = [
    defaultCiphers[0],
    // Swap the 2nd & 3rd ciphers:
    defaultCiphers[2],
    defaultCiphers[1],
    ...defaultCiphers.slice(3)
].join(':');

request = require('https').get('https://google.com', {
    ciphers: shuffledCiphers
}).on('response', (res) => {
    console.log(res.statusCode);
});
// source: https://httptoolkit.tech/blog/tls-fingerprinting-node-js/

More information about this can be found here: https://httptoolkit.tech/blog/tls-fingerprinting-node-js/

user_94997850
Related