max.in.flight.requests.per.connection kafka producer configuration

user_13476284 0
Viewed 928

I have a question regarding the max.in.flight.requests.per.connection kafka producer configuration. Is this configuration applies on per message or all messages in a batch sent to the broker ?

for example, if my batch size is 16KB and size of each message is 1 KB then batch will have 16 messages.

if max.in.flight.requests.per.connection value is 1 then will producer send one message from the batch and wait for the acknowledgment before sending next message from the batch ?

or

producer will send all the messages in the batch and wait for the acknowledgment before sending the next batch ?

apache-kafka
1 Answers

max.in.flight.requests.per.connection (pipelining)

This property is configured on the producer max.in.flight.requests.per.connection property. This property applies to the batch of messages not to individual messages.

max.in.flight.requests.per.connection=1 means one batch of record will be sent one time and wait till the response is not received from the broker.

The maximum number of unacknowledged requests the client will send on a single connection before blocking. If this setting is greater than 1, pipelining is used when the producer sends the grouped batch to the broker.

This improves throughput, but if there are failed sends there is a risk of out-of-order delivery due to retries (if retries are enabled).

Note also that excessive pipelining reduces throughput.

user_44224790
Related