Reading response body beyond content-length

Viewed 320

I'm working against an HTTP server that sometimes returns a body that is longer than the response's Content-length header. For example, curl shows me this:

curl -v -k -H "Cookie: VerySecretCookie" https://fqdn/path

[...]

< HTTP/1.1 200 200
< Date: Thu, 01 Jul 2021 10:45:32 GMT
[...]
< X-Frame-Options: SAMEORIGIN
<
* Excess found in a non pipelined read: excess = 83, size = 2021, maxdownload = 2021, bytecount = 0

I'm trying to issue a similar request via axios and read the entire body, even beyond the Content-length. However, it seems to stop after Content-length bytes have been read.

This is the config I'm using:

const config: AxiosRequestConfig = {
  method: 'GET',
  responseType: 'stream',
  maxRedirects: 0,
  decompress: false,
  timeout: 60000,
};

And I then retrieve the response via:

private async streamToString(stream: Stream): Promise<string> {
  return new Promise((res, rej) => {
    let data = '';
    stream.on('end', () => {
      res(data);
    });

    stream.on('error', (err) => {
      rej(err);
    });

    stream.on('data', (chunk) => {
      data += chunk;
    });
  });
}

const data = await this.streamToString(response.data);

From a quick glance at axios' code, I couldn't find a place where it inspects the Content-length header, so perhaps this is done by Node's http? Is there a way to tell it to ignore Content-length and just read the entire stream until it closes (assuming HTTP keepalive is not used)?

0 Answers
Related