Serving multiple index files with gatsby & aspnet

I'm using the dotnet core react template with gatsby as the frontend. I have a routing issue with aspnet core. If a file is not found, aspnet serves the index.html from the root. This would be fine in most cases, but gatsby generates multiple index files - one for each "page":

wwwroot:

• index.html
• second-page/
  • index.html
• third-page/
  • index.html

This works fine when the user lands on the root (/) and navigates client-side. On a refresh inside a page/folder (e.g. website.com/second-page/) the aspnet returns the index.html from the root.

So, I'd like to have the index file of the corresponding folder.

Is this possible with some configuration or do I have to write a custom middleware?

Here's my Configure method from Startup.cs:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
    {
        if (env.IsDevelopment())
        {
            app.UseDeveloperExceptionPage();
        }
        else
        {
            app.UseExceptionHandler("/Error");
            app.UseHsts();
        }
    
        app.UseHttpsRedirection();
        app.UseStaticFiles();
        app.UseSpaStaticFiles();
    
        app.UseRouting();
    
        app.UseEndpoints(endpoints =>
        {
            endpoints.MapControllerRoute(
                name: "default",
                pattern: "{controller}/{action=Index}/{id?}");
        });
    
        app.UseSpa(spa =>
        {
            spa.Options.SourcePath = "ClientApp";
    
            if (env.IsDevelopment())
            {
                spa.UseProxyToSpaDevelopmentServer("http://localhost:8000/");
            }
        });
    }