We have built an application using spring boot for enterprise reasons and using the spring Kafka project to publish Kafka messages.
I have been testing producing Kafka events and noticed that spring is blocking the HTTP thread till the Kafka thread is complete.
try {
ListenableFuture<SendResult<String, OnboardingSigningNotificationEvent>> send = this.kafkaTemplate.send(TOPIC, notificationEvent);
send.addCallback(new ListenableFutureCallback<SendResult<String, OnboardingSigningNotificationEvent>>() {
@Override
public void onSuccess(SendResult<String, OnboardingSigningNotificationEvent> sendResult) {
log.info("Successfully published onboarding signing notification event with record metadata timestamp: {}", sendResult.getRecordMetadata().timestamp());
}
@Override
public void onFailure(Throwable throwable) {
log.error("Failure while publishing kafka event : {}", throwable.getMessage());
}
});
} finally {
kafkaTemplate.flush();
}
This contradicts what is said in this documentation. https://docs.spring.io/spring-kafka/docs/current/reference/html/#sending-messages
Notice that the send methods return a ListenableFuture. You can register a callback with the listener to receive the result of the send asynchronously.
Based on the documentation I would expect the producer call to be made asynchronously and the main http thread to not block waiting for it, but that's not what I see in my tests. Even if I block the kafka thread for a few seconds the caller http thread is being blocked.
Can you please explain whether it is on purpose and are there any config to override this behavior.