Kaka Stream : Processor API -> InvalidTxnStateException : The producer attemted a transactional operation in an Invalid State

Viewed 16

I am using Kafka Stream with processor API(Topology) with Processing.guarantee=exactly_once_v2 When I am trying to deploy application on UAT environment, application started successfully, but When I restart the application it throw below exception for one of the partition

Exception : org.apache.kafka.common.errors.InvalidTxnStateException:The producer attempted a transactional operation in an Invalid State

After this I have restarted the application and it is throwing below error and application stopped.

Error : The following partitions still have unstable offsets which are not cleared on the broker side : ["SOURCE TOPIC PARTITION NAME"], this could be either transactional offsets waiting for completion, or normal offsets waiting for replication after appending to local log

High-level Architecture of application :

  1. Consume message from Source topic
  2. Add consumed message in State store.(Change-log topic is created manually)
  3. Scheduler will run certain interval, aggregate message from State Store and forward it to sink processor.
  4. context.commit().

**Note :

  1. Working as expected in Dev environment.
  2. Kafka Stream Configuration is same as Dev.
  3. Tried with latest version of Kafka Stream.**

Any help would be appreciated.

0 Answers
Related