Our kafka system crashed because no disk space was available. The consumers are Spring boot application which are using the Kafka Streams API. Now every consumer application shows the following error:
java.io.FileNotFoundException: /tmp/kafka-streams/908a79bc-92e7-4f9c-a63a-5030cf4d3555/streams.device-identification-parser/0_48/.checkpoint.tmp (No such file or directory)
This exception occurred exactly after the kafka server was restarted. If we restart the application, the service starts at offset 0 to replay all message from the consumed topic. This is fatal.
We checked the logs: On each partition there is no information (example line, we have 60 partitions):
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG CONSUMER-ID HOST CLIENT-ID
device-identification.received 0 - 0 - streams.device-identification-parser-3e81ef05-b9e3-495c-85b3-7e2912f00578-StreamThread-1-consumer-ca723291-3258-46c0-8b23-233bdc1e1191 /XX.X.X.X streams.device-identification-parser-3e81ef05-b9e3-495c-85b3-7e2912f00578-StreamThread-1-consumer
Can anyone help me, to restore the offset information, that the service starts from the last offset?