I'm working against an HTTP server that sometimes returns a body that is longer than the response's Content-length header. For example, curl shows me this:
curl -v -k -H "Cookie: VerySecretCookie" https://fqdn/path
[...]
< HTTP/1.1 200 200
< Date: Thu, 01 Jul 2021 10:45:32 GMT
[...]
< X-Frame-Options: SAMEORIGIN
<
* Excess found in a non pipelined read: excess = 83, size = 2021, maxdownload = 2021, bytecount = 0
I'm trying to issue a similar request via axios and read the entire body, even beyond the Content-length. However, it seems to stop after Content-length bytes have been read.
This is the config I'm using:
const config: AxiosRequestConfig = {
method: 'GET',
responseType: 'stream',
maxRedirects: 0,
decompress: false,
timeout: 60000,
};
And I then retrieve the response via:
private async streamToString(stream: Stream): Promise<string> {
return new Promise((res, rej) => {
let data = '';
stream.on('end', () => {
res(data);
});
stream.on('error', (err) => {
rej(err);
});
stream.on('data', (chunk) => {
data += chunk;
});
});
}
const data = await this.streamToString(response.data);
From a quick glance at axios' code, I couldn't find a place where it inspects the Content-length header, so perhaps this is done by Node's http? Is there a way to tell it to ignore Content-length and just read the entire stream until it closes (assuming HTTP keepalive is not used)?