I am using Kafka Stream with processor API(Topology) with Processing.guarantee=exactly_once_v2 When I am trying to deploy application on UAT environment, application started successfully, but When I restart the application it throw below exception for one of the partition
Exception : org.apache.kafka.common.errors.InvalidTxnStateException:The producer attempted a transactional operation in an Invalid State
After this I have restarted the application and it is throwing below error and application stopped.
Error : The following partitions still have unstable offsets which are not cleared on the broker side : ["SOURCE TOPIC PARTITION NAME"], this could be either transactional offsets waiting for completion, or normal offsets waiting for replication after appending to local log
High-level Architecture of application :
- Consume message from Source topic
- Add consumed message in State store.(Change-log topic is created manually)
- Scheduler will run certain interval, aggregate message from State Store and forward it to sink processor.
- context.commit().
**Note :
- Working as expected in Dev environment.
- Kafka Stream Configuration is same as Dev.
- Tried with latest version of Kafka Stream.**
Any help would be appreciated.