Why does FullHttpResponse reference counting behave differently dependeing on the underlying ByteBuf implementation?

Question

I'm having trouble with leaks on an EmbeddedChannel I use to parse a raw http response into a FullHttpResponse. During analysis, I noticed that the reference counting of the ByteBuf I send into the channel as "input" behaves differently depending on its implementation, which seems odd.

For reference, here is the method I use to parse the raw input:

private FullHttpResponse decodeHttpResponse(ByteBuf responseBuff) {
    EmbeddedChannel channel = new EmbeddedChannel(new HttpResponseDecoder(), new HttpObjectAggregator(99999999));
    try {
        channel.writeInbound(responseBuff);
        channel.flush();
        channel.finish();
        return channel.readInbound();
    } finally {
        channel.close();
        channel.finishAndReleaseAll();
    }
}

Let's use a simple ByteBuf I create from an http response I previously dumped into a file and do some testing, here's my output (the numbers are the reference counts of the object):

>>>---------- Test with ByteBuf ------------<<<
>>> [buf]  : 1
buf.retain()
>>> [buf]  : 2
resp = decodeHttpResponse(cbuf) => io.netty.handler.codec.http.HttpObjectAggregator$AggregatedFullHttpResponse
>>> [buf]  : 2                  // [1]
>>> [resp] : 1                  // 
resp.retain()
>>> [buf]  : 2
>>> [resp] : 2
resp.release()
>>> [buf]  : 2
>>> [resp] : 1
resp.release()                  //
>>> [buf]  : 1                  // [2]
>>> [resp] : 0                  //

[1]: When we create the FullHttpResponse, the refCount on the input ByteBuf is not changed.
[2]: When the FullHttpResponse gets deallocated (refCount reaches 0), it propagates one single release to the underlying ByteBuf.

OK, now let's wrap that input ByteBuf into a CompositeByteBuf and use that to build our FullHttpResponse:

>>>---------- Test with CompositeByteBuf ------------<<<
>>> [buf]  : 1
buf.retain()
>>> [buf]  : 2
cbuf = buf.alloc().compositeBuffer()
>>> [buf]  : 2
>>> [cbuf] : 1
cbuf.addComponent(true, buf)
>>> [buf]  : 2
>>> [cbuf] : 1
cbuf.retain()
>>> [buf]  : 2
>>> [cbuf] : 2
resp = decodeHttpResponse(cbuf) => io.netty.handler.codec.http.HttpObjectAggregator$AggregatedFullHttpResponse
>>> [buf]  : 2              //
>>> [cbuf] : 1              // [1]
>>> [resp] : 1              //
resp.retain()
>>> [buf]  : 2
>>> [cbuf] : 1
>>> [resp] : 2
resp.release()
>>> [buf]  : 2
>>> [cbuf] : 1
>>> [resp] : 1
resp.release()
>>> [buf]  : 2              //
>>> [cbuf] : 1              // [2]
>>> [resp] : 0              //

[1]: When we create the FullHttpResponse, the refCount on the input ByteBuf is being decreased by 1.
[2]: When the FullHttpResponse gets deallocated (refCount reaches 0), it does not propagate to the underlying CompositeByteBuf.

This difference in behavior is confusing and makes correct reference counting hard (this is a simplified example, in a real application, the entanglement of layered buffers makes it even harder).

Can someone explain why the above behaves differently with ByteBuf and CompositeByteBuf or if there's something I can do to make the behavior consistent and predictable? Or is this maybe even a bug that should be addressed?

Why does FullHttpResponse reference counting behave differently dependeing on the underlying ByteBuf implementation?

0 Answers