I run into issues when starting two streams applications from the same machine. It has to do with the Cooperative rebalancing protocol. For some reason when the second one comes up the first one crashes with:
[streams-StreamThread-1] INFO org.apache.kafka.streams.processor.internals.StreamThread - stream-thread [streams-stream-1-StreamThread-1] State transition from PENDING_SHUTDOWN to DEAD
[streams-StreamThread-1] INFO org.apache.kafka.streams.KafkaStreams - stream-client [streams-app.id-1] State transition from RUNNING to ERROR
[streams-StreamThread-1] ERROR org.apache.kafka.streams.KafkaStreams - stream-client [streams-1] All stream threads have died. The instance will be in error state and should be closed.
[streams-1] INFO org.apache.kafka.streams.processor.internals.StreamThread - stream-thread [streams-StreamThread-1] Shutdown complete
[streams-StreamThread-1] ERROR app.id - Uncaught exception in Streams thread (streams-StreamThread-1)
java.lang.IllegalStateException: Assignor supporting the COOPERATIVE protocol violates its requirements
at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.validateCooperativeAssignment(ConsumerCoordinator.java:668)
I have tried using a separate state.dir on the two instances but that didn’t seem to work for me either.
Can be ignored, see first comment: Further, I notice the same behavior when trying to start two consumers with the CooperativeStickyAssignor configured. Is there something with regards to Cooperative Rebalancing configuration that I am missing?