MSAL.NET OBO refresh token problems

I am trying to implement an OBO flow through to the graph API on a middle-tier API (.NET 5.0) using MSAL.NET. I'm running into two frustrating problems, and I can't find anyone having similar problems, so I think I'm misunderstanding something!

Problem 1: Whenever I call MSAL's GetAccountAsync, it always returns null when there should be an account loaded.

Problem 2: Whenever I call MSAL's AcquireTokenSilent, I always get the error "No refresh token found in the cache." even though I got one.

Here's what I have:

Once the web app authenticates, it passes through the token to a graph auth endpoint on the API:

            var authenticationResult = await ClaimHelper.ClientApplication.AcquireTokenByAuthorizationCode(GraphHelpers.BasicGraphScopes, context.Code).ExecuteAsync();

            var apiUserSession = await CouncilWiseAPIHelper.APIClient.Graph.AuthoriseUserAsync(authenticationResult.AccessToken);

which seems to work fine, and passes through a JWT to the API auth endpoint. The API implements an MSAL Confidential Client application and uses the SetBeforeAccess/SetAfterAccess token cache methods to save the cache to a database.

         _msalClient = ConfidentialClientApplicationBuilder.Create(_graphConfig.ClientId)
            .WithAuthority(AadAuthorityAudience.AzureAdMultipleOrgs)
            .WithClientSecret(_graphConfig.ClientSecret)
            .Build();

        SetSerialiser(serialiser);

        public void SetSerialiser(MSALTokenCacheSerialiser serialiser)
        {
            _msalClient.UserTokenCache.SetBeforeAccessAsync(serialiser.BeforeAccessCallbackAsync);
            _msalClient.UserTokenCache.SetAfterAccessAsync(serialiser.AfterAccessCallbackAsync);
        }

And the serialiser methods look like this:

public async Task BeforeAccessCallbackAsync(TokenCacheNotificationArgs notification)
        {
            GraphUserTokenCache tokenCache = await _graphUserTokenCacheRepository.GetByUserIdentifier(notification.SuggestedCacheKey);
            if (tokenCache == null)
            {
                tokenCache = await _graphUserTokenCacheRepository.Get(notification.SuggestedCacheKey);
            }

            if (tokenCache != null)
            {
                notification.TokenCache.DeserializeMsalV3(tokenCache.Value);
            }
        }

        public async Task AfterAccessCallbackAsync(TokenCacheNotificationArgs notification)
        {
            if (!notification.HasTokens)
            {
                // Delete from the cache
                await _graphUserTokenCacheRepository.Delete(notification.SuggestedCacheKey);
            }

            if (!notification.HasStateChanged)
            {
                return;
            }
            GraphUserTokenCache tokenCache;
            if (notification.SuggestedCacheKey == notification.Account.HomeAccountId.Identifier)
            {
                tokenCache = await _graphUserTokenCacheRepository.GetByUserIdentifier(notification.SuggestedCacheKey);
            }
            else
            {
                tokenCache = await _graphUserTokenCacheRepository.Get(notification.SuggestedCacheKey);
            }
            if (tokenCache == null)
            {
                var cache = notification.TokenCache.SerializeMsalV3();
                tokenCache = new GraphUserTokenCache
                {
                    Id = notification.SuggestedCacheKey,
                    AccountIdentifier = notification.Account.HomeAccountId.ToString(),
                    Value = cache
                };

                await _graphUserTokenCacheRepository.Add(tokenCache);
            }
            else
            {
                await _graphUserTokenCacheRepository.Update(tokenCache.Id, notification.TokenCache.SerializeMsalV3());
            }
        }

I can see the token BeforeAccess and AfterAccess methods being called, and I can see the caches being created in the database (encryption has been removed while I'm trying to track down this issue). If I inspect the serialised token cache being saved, it NEVER has a refresh token populated, but if I inspect the requests with fiddler I can see a refresh token was indeed provided. Finally, here is the code for retrieving the access token which is called whenever a graph request is made:

public async Task<AuthenticationResult> GetAccessToken(string accountId, string jwtBearerToken)
        {
            try
            {
                IAccount account = null;
                if (accountId.IsNotNullOrEmpty())
                {
                    account = await _msalClient.GetAccountAsync(accountId);
                }

                var scope = _graphConfig.Scopes.Split(' ');

                if (account == null)
                {
                    var result = await _msalClient.AcquireTokenOnBehalfOf(scope,
                        new UserAssertion(jwtBearerToken))
                        .ExecuteAsync();

                    return result;
                }
                else
                {
                    var result = await _msalClient.AcquireTokenSilent(scope, account)
                    .ExecuteAsync();

                    return result;
                }
            }
            catch (MsalClientException ex)
            {
                ex.CwApiLog();
                return null;
            }
            catch(Exception ex)
            {
                ex.CwApiLog();
                return null;
            }
        }

When it's called with the jwtBearerToken, it will successfully call AcquireTokenOnBehalfOf() and the token is cached and a result returned, but when I come back to retrieve the account via GetAccountAsync() it always returns null even though I can see the token cache was loaded in BeforeAccessCallbackAsync().

Also, even if I call AcquireTokenSilent() immediately after acquiring the obo token with the account it just returned, I will get an exception saying there is no refresh token in the cache.

I am totally lost on what I'm doing wrong here, any help would be greatly appreciated.