asp.net core api - WebApplicationFactory - override 'configure'/ 'WebApplicationBuilder.build()'

I'm currently trying to create Integration-Tests for my asp.net core API Project using dotnet6. For Production im using a MsSql-Db and for the Integration-Tests I want to use a SqlLite-InMemory-Db. I've got my Tests running and working, however theres one point in my code which I really don't like.

What I did so far :

• Created the API Project including Endpoints to test

• Created the Test-Project Adding the "InternalsVisibleTo" Attribute to my "API.csproj"

• Creating a Custom WebApplicationFactory, which also replaces the prod-db with the SqlLite - InMemory db and migrates the required test data

    internal class CustomTestApiFactory : WebApplicationFactory<Program>
    {
        private readonly string _azureAdminGruppenId = "<Guid>";
  
        protected override void ConfigureWebHost(IWebHostBuilder builder)
        {
  
            builder.UseEnvironment("IntegrationTests");
            builder.ConfigureServices(services =>
            {
                ReplaceDefaultSqlProviderForInMemorySqlLite(services);
                services.AddAuthentication("Test")
                    .AddScheme<AuthenticationSchemeOptions, TestAuthHandler>("Test", options => { });
  
                var communicator = Substitute.For<IMicrosoftGraphCommunicator>();
                communicator.GetCurrentUserGroupIds().Returns(new[] { _azureAdminGruppenId });
                services.AddSingleton(communicator);
  
                // Build the service provider.
                var sp = services.BuildServiceProvider();
                InsertIntegrationTestData(sp);
            });
        }
  
        private void ReplaceDefaultSqlProviderForInMemorySqlLite(IServiceCollection services)
        {
            var dbProviderToRemove = services.SingleOrDefault(d => d.ServiceType == typeof(DbContextOptions<ApplicationDbContext>));
            if (dbProviderToRemove != null)
            {
                services.Remove(dbProviderToRemove );
            }
  
            var connectionStringBuilder = new SqliteConnectionStringBuilder { DataSource = ":memory:" };
            var con = new SqliteConnection(connectionStringBuilder.ToString());
            con.Open();
            services
                .AddEntityFrameworkSqlite()
                .AddDbContextPool<ApplicationDbContext>(options =>
                {
                    options.UseSqlite(con);
                });
  
        }
  
        private void InsertIntegrationTestData(ServiceProvider sp)
        {
            using var scope = sp.CreateScope();
            try
            {
                var scopedServices = scope.ServiceProvider;
                var appDb = scopedServices.GetRequiredService<ApplicationDbContext>();
                appDb.Database.EnsureDeleted();
                appDb.Database.EnsureCreated();
                appDb.SeedData();
            }
            catch (Exception e)
            {
                Console.WriteLine(e);
                throw new SystemException("Cannot insert integration - testdata");
            }
        }
    }
  

However, in my "API-program.cs" I had to do the following

var app = builder.Build();
using (var scope = app.Services.CreateScope())
{
    var dbContext = scope.ServiceProvider.GetService<ApplicationDbContext>();
    if (builder.Environment.EnvironmentName is "Test" or "Development" or "Production")
    {
        dbContext?.Database.Migrate();
    }
    var uow = scope.ServiceProvider.GetService<IUnitOfWork>();

    if (uow != null)
    {
        await (new PermissionSeeder(uow)).Seed();
        await (new AdminSeeder(uow, builder.Configuration)).Seed();
    }
}

app.UseSession();
app.UseAuthentication();
app.UseMiddleware<ExceptionMiddleware>();
app.UseRouting();
app.UseAuthorization();
app.UseEndpoints(
    endpoints =>
    {
        endpoints.MapDefaultControllerRoute();
    }
);
app.Run();

I had to add a condition for the "dbContext?.Database.Migrate();" because It would fail on the SqlLite-InMemory. But I feel like I should not change the actual code for my testing. Is there any way to overwrite the part after "builder.Build()" (basically the code that would be inside the "Configure" Method in dotnet 5). I'd be fine to reimplement the code after "builder.Build()" for the integration test-server.

Thanks in advance!