ISPN000580: Failed to migrate persisted data - upgrading to jboss/keycloak 13.0.1

Viewed 412

I am trying to upgrade jboss/keycloak 6.0.1 to 13.0.1 which is running as StatefulSet in k8s. I have converted my standalone-ha.xml and I am getting the following error:

[0m[32m13:05:34,632 DEBUG [org.infinispan.persistence.manager.PersistenceManagerImpl] (ServerService Thread Pool -- 68) PersistenceManagerImpl encountered an exception during startup of stores: java.util.concurrent.CompletionException: org.infinispan.persistence.spi.PersistenceException: ISPN000580: Failed to migrate persisted data.
at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:314)
at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:319)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1739)
at org.jboss.threads@2.4.0.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
at org.jboss.threads@2.4.0.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1348)
at org.jboss.as.clustering.common@23.0.2.Final//org.jboss.as.clustering.context.ContextReferenceExecutor.execute(ContextReferenceExecutor.java:49)
at org.jboss.as.clustering.common@23.0.2.Final//org.jboss.as.clustering.context.ContextualExecutor$1.run(ContextualExecutor.java:70)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: org.infinispan.persistence.spi.PersistenceException: ISPN000580: Failed to migrate persisted data.
at org.infinispan@11.0.9.Final//org.infinispan.persistence.file.SingleFileStore.migrateFromV1(SingleFileStore.java:373)
at org.infinispan@11.0.9.Final//org.infinispan.persistence.file.SingleFileStore.start(SingleFileStore.java:160)
at org.infinispan@11.0.9.Final//org.infinispan.persistence.support.NonBlockingStoreAdapter.lambda$start$0(NonBlockingStoreAdapter.java:108)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
... 7 more
Caused by: protostream.com.google.protobuf.InvalidProtocolBufferException: Protocol message contained an invalid tag (zero).
at org.infinispan.protostream@4.3.5.Final//protostream.com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:102)
at org.infinispan.protostream@4.3.5.Final//protostream.com.google.protobuf.CodedInputStream$ArrayDecoder.readTag(CodedInputStream.java:627)
at org.infinispan.protostream@4.3.5.Final//org.infinispan.protostream.impl.RawProtoStreamReaderImpl.readTag(RawProtoStreamReaderImpl.java:45)
at org.infinispan.protostream@4.3.5.Final//org.infinispan.protostream.WrappedMessage.readMessage(WrappedMessage.java:275)
at org.infinispan.protostream@4.3.5.Final//org.infinispan.protostream.ProtobufUtil.fromWrappedByteArray(ProtobufUtil.java:162)
at org.infinispan@11.0.9.Final//org.infinispan.marshall.persistence.impl.PersistenceMarshallerImpl.objectFromByteBuffer(PersistenceMarshallerImpl.java:155)
at org.infinispan@11.0.9.Final//org.infinispan.persistence.file.SingleFileStore.migrateFromV1(SingleFileStore.java:333)
... 10 more

Any idea how to tackle this error?

3 Answers

Do you try to update your Keycloak system while still running any instances? This is not supported by Keycloak. It's recommended to shut down all instances and start migration with one instance, when upgrading major versions!

Between Keycloak 6 and 13 the underlying Infinispan versions have changed and thus at some point also the de-/serialization mechanism. Most possibly that's the cause for your errors.

The error was due the cache. In order to make it work the cache will need to be migrated from the syntax that keycloak 6 uses to 13. In our case, we preferred to start with a new cache.

Official solution from Red Hat:

Delete saved web session data by removing all of the the contents of the directory $JBOSS_HOME/standalone/data/infinispan/web

Related