Using ASP.Net Core Identity and Azure authentication together in Blazor

I have a server-side Blazor app which is successfully integrated with Azure for authentication. The standard approach worked here, I modified Startup.cs as follows:

services.AddAuthentication(OpenIdConnectDefaults.AuthenticationScheme)
                    .AddMicrosoftIdentityWebApp(Configuration.GetSection("AzureAd"));
            services.AddControllersWithViews()
                    .AddMicrosoftIdentityUI();
            services.AddAuthorization(options => {
                // By default, all incoming requests will be authorized 
                // according to the default policy
                options.FallbackPolicy = options.DefaultPolicy;
            });

And then I check for authentication in MainLayout.razor as follows:

    protected override async Task OnInitializedAsync()
    {
        AuthenticationState authState = await AuthenticationStateProvider.GetAuthenticationStateAsync();
        System.Security.Claims.ClaimsPrincipal user = authState.User;
        System.Security.Principal.IIdentity userIdent = user.Identity;
        if (userIdent.IsAuthenticated)
        {
            doMoreStuff(user);
        }
        ...
    }

I would like to ALSO enable authorization via Identity with custom UserStore and RoleStore implementations (NOT using Entity framework core). I started to implement user store and role store, and added them to Startup.cs:

IdentityBuilder builder = services.AddIdentity<MyUser, MyRole>()
                .AddDefaultTokenProviders();            
builder.Services.AddScoped(typeof(IUserStore<>).MakeGenericType(builder.UserType), typeof(MyUserManager));
builder.Services.AddScoped(typeof(IRoleStore<>).MakeGenericType(builder.RoleType), typeof(MyRoleManager));

But as soon as I add these custom Identity additions, I immediately see the code in MainLayout calling AuthorizationStateProvider.GetAuthenticationStateAsync() now returns null.

Any suggestions on how to implement BOTH custom Identity AND the Azure authentication? They don't want to play well together for me.