How to use BiometricPrompt with CryptoObject to get KeyStore access for both Decryption and Encryption at the same time?

PROBLEM DESCRIPTION: When accessing the Android KeyStore with BiometricPrompt authentication, the BiometricPrompt appears every time when I have to perform read or write operation to the KeyStore. I am looking for a solution to authenticate only once, and then manipulate data in the keystore as I wish, in a similar way as it is done on iOS KeyChain.

I have implemented biometric authentication for Android so that I can store app API AuthRefreshToken in the application KeyStore secured by the biometric authentication, by calling setUserAuthenticationRequired(true) on the KeyStore params builder (see below). I have followed examples from Google (https://github.com/android/security-samples/tree/main/BiometricLoginKotlin) and other developers and have made solution working successfully. I am now trying to resolve the problem described above for second working day without success, and now considering using BiometricPrompt without CryptoObject, which would be a big disappointment. I suspect there is a way to authenticate once for a period of time, perhaps by setting paramsBuilder.setUserAuthenticationValidityDurationSeconds(30), but I am unable to achieve the intended result.

To get access to the KeyStore and read the API AuthRefreshToken, I use this code:

biometricPrompt = BiometricPromptUtils.createBiometricPrompt(this, ::decryptServerTokenFromStorage)
val promptInfo = BiometricPromptUtils.createPromptInfo(this)
biometricPrompt.authenticate(promptInfo, BiometricPrompt.CryptoObject(cipher))

fun decryptServerTokenFromStorage(authResult: BiometricPrompt.AuthenticationResult) {
    ciphertextWrapper?.let { textWrapper ->
        authResult.cryptoObject?.cipher?.let {
            val authRefreshToken = cryptographyManager.decryptData(textWrapper.ciphertext, it)
            // Use authRefreshToken to get authToken from the API
            // The API returns new authRefreshToken which has to be saved back to the KeyStore
        }
    }
}

Everything works fine, and I get decrypted data. However, after each authentication on the app API using AuthRefreshToken, the token changes and I have to immediately save it back to the KeyStore. When this happens, I use below code, which displays the BiometricPrompt again. This causes the UI flow to show the BiometricPrompt twice:

biometricPrompt = BiometricPromptUtils.createBiometricPrompt(this, ::encryptServerTokenToStorage)
val promptInfo = BiometricPromptUtils.createPromptInfo(this)
biometricPrompt.authenticate(promptInfo, BiometricPrompt.CryptoObject(cipher))

fun encryptServerTokenToStorage(authResult: BiometricPrompt.AuthenticationResult) {
    authResult.cryptoObject?.cipher?.apply {
            SampleAppUser.refreshAuthToken?.let { refreshAuthToken ->
                Log.d(TAG, "The token from server is $refreshAuthToken")
                val encryptedServerTokenWrapper = cryptographyManager.encryptData(refreshAuthToken, this)
                // Now save encrypted authRefreshToken together with initializationVector in the app prefs for future authentications
            )
        }
    }
}

How can I authenticate at once with the BiometricPrompt so that I have full read/write access to the KeyStore for, let say, 1 minute or a longer interval without calling the BiometricPrompt multiple times?

I have tried different approaches and tried recreating the Cipher or reinitializing it for a different purpose, however in all these and similar attempts I am getting Javax.Crypto.IllegalBlockSizeException with message 'Key user not authenticated'

The keystore initialization is as follows:

    // If Secretkey was previously created for that keyName, then grab and return it.
    val keyStore = KeyStore.getInstance(ANDROID_KEYSTORE)
    keyStore.load(null) // Keystore must be loaded before it can be accessed
    keyStore.getKey(keyName, null)?.let { return it as SecretKey }

    // if you reach here, then a new SecretKey must be generated for that keyName
    val paramsBuilder = KeyGenParameterSpec.Builder(
        keyName,
        KeyProperties.PURPOSE_ENCRYPT or KeyProperties.PURPOSE_DECRYPT
    )
    paramsBuilder.apply {
        setBlockModes(ENCRYPTION_BLOCK_MODE) // KeyProperties.BLOCK_MODE_GCM
        setEncryptionPaddings(ENCRYPTION_PADDING) // KeyProperties.ENCRYPTION_PADDING_NONE
        setKeySize(KEY_SIZE) // 256
        setUserAuthenticationRequired(true) // This is required for BiometricPrompt to work properly
    }

    val keyGenParams = paramsBuilder.build()
    val keyGenerator = KeyGenerator.getInstance(KeyProperties.KEY_ALGORITHM_AES, "AndroidKeyStore")
    keyGenerator.init(keyGenParams)
    return keyGenerator.generateKey()
}