I am going through the documentation and it is little confusing about the parameter "max.in.flight.requests.per.connection"
The maximum number of unacknowledged requests the client will send on a single connection before blocking. Note that if this setting is set to be greater than 1 and there are failed sends, there is a risk of message re-ordering due to retries (i.e., if retries are enabled).
The phrase "unacknowledged requests" refers to per producer or per connection or per client ?
It's per partition. Kafka internally might multiplex connections (e.g. to send several requests using a single connect for different topics/partitions that are handled by the same broker), or have an individual connection per partition, but these are performance concerns which are mostly dealt within the client.
The documentation of retries, sheds some more light (and clarifies that is per partition)
Setting a value greater than zero will cause the client to resend any record whose send fails with a potentially transient error. Note that this retry is no different than if the client resent the record upon receiving the error. Allowing retries without setting max.in.flight.requests.per.connection to 1 will potentially change the ordering of records because if two batches are sent to a single partition, and the first fails and is retried but the second succeeds, then the records in the second batch may appear first. Note additionally that produce requests will be failed before the number of retries has been exhausted if the timeout configured by delivery.timeout.ms expires first before successful acknowledgement. Users should generally prefer to leave this config unset and instead use delivery.timeout.ms to control retry behavior.
This is a setting per connection, per broker. If you have a producer, then internally it uses a Sender Thread that dispatches batches from the RecordAccumulator to the broker (in simpler words : sends messages). This sender thread is allowed to have a max of ${max.in.flight.requests.per.connection} requests that it has not yet received acknowledgements from the broker. Think about this way: a sender does some operations in typical processing.
Drain batches -> Make Requests -> Pool Connections -> Fire Callbacks.
So at some point (Pool Connections) it can send a request to the broker, but not wait for a response, it will check for the response in the next cycle. It can have such unacknowledged requests, up to that max.in.flight.requests.per.connection value.