I'm working on a MITM proxy on Node using typescript. I'm trying to decode responses that are encoded using gzip. I don't want to just remove accept-encoding header as I saw in some answers here on SO. I want to use zlib to decode the response body, but for some reason when I try to load a page that uses gzip (like github.com for example) the page doesn't load (images, colors, texts, etc.). My decompress is not working and I don't know why. The code that I'm using to decompress the response body is the following:
NOTE: serverResponse is the response from the server that I (as the proxy) am connecting to (github.com for example) and proxyResponse is the response from me (the proxy) to the client that started the request
protected async receiveResponse(serverResponse: http.IncomingMessage, proxyResponse: http.ServerResponse) {
const contentEncoding = serverResponse.headers["content-encoding"]
let responseContent: http.IncomingMessage | zlib.Gunzip = serverResponse
if (contentEncoding && contentEncoding.toLowerCase().includes("gzip")) {
responseContent = zlib.createGunzip()
serverResponse.pipe(responseContent)
delete serverResponse.headers["content-encoding"]
}
let responseBody: Buffer
try {
responseBody = await this.collectMessageBody(responseContent)
} catch (error) {
console.log(error)
return
}
proxyResponse.writeHead(serverResponse.statusCode!, serverResponse.headers)
proxyResponse.write(responseBody)
proxyResponse.end()
}
private collectMessageBody(stream: http.IncomingMessage | zlib.Gunzip): Promise<Buffer> {
return new Promise<Buffer>((resolve, reject) => {
let bodyBuffers: Buffer[] = []
stream.on('data', chunk => bodyBuffers.push(chunk))
stream.on('end', () => resolve(Buffer.concat(bodyBuffers)))
stream.on('error', error => reject(error))
})
}
It follows the same strategy that most answers here on the SO, but I don't know why mine is not working.
OBS: This is an open-source project, so the entire file can be found here: https://github.com/olmps/web-sniffer/blob/master/src/server.ts
I've simplified the logic removing unnecessary stuff to post here.
As an example, when loading github.com that uses gzip to compress its content, I get the following result when the proxy is on:
