Why does `UseAuthentication` have to be placed after `UseRouting` and not before?

After posting this question I've opened an issue about routing on github, and one about localization, hoping for more information. And though not everything was answered in a direct way, it helped me to find an answer to this question.

After reading the comment of David Fowler:

UseAuthorization() -> Will look at the populated user and the current endpoint to determine if an authorization policy needs to be applied.

it occured to me that there is no problem with UseAuthorization. It's meant for endpoints, so I don't need it to protect the folders. It also explains why this statement only makes sense after the UseEndpoints statement.

For a complete picture of my configuration, I have one policy provider (includes policies), one url rewriter (like UseDefaultFiles) and middleware that protects certain folders.

My conclusion is that I can use the following order, which is almost the same as documented:

// Identify the user. The only statement that is not in the order as documented
app.UseAuthentication();

// Middleware that adds policies
app.UsePolicyProvider();
// Protect the folder by policy
app.UseProtectFolder(new ProtectFolderOptions { Path = "/p", PolicyName = "admin" });
// URL rewriter for serving tenant specific files
app.UseTenantStaticFiles();

// Serve the static files
app.UseStaticFiles();

app.UseCookiePolicy();
app.UseCors();

app.UseRouting();
app.UseRequestLocalization();
app.UseAuthorization();
app.UseEndpoints();

Two remarks about the order:

1. UseRequestLocalization only works after UseRouting
2. UseStaticFiles doesn't work after UseRouting when a URL rewriter, like UseDefaultFiles, is involved.

One thing I wanted to add to @Ruard's answer regarding the "Why UseRouting before UseAuth" part of this question is this excerpt from Overview of ASP.NET Core authentication:

When using endpoint routing, the call to UseAuthentication must go:

• After UseRouting, so that route information is available for authentication decisions.
• Before UseEndpoints, so that users are authenticated before accessing the endpoints.

I was still curious what route information was necessary before UseAuthentication() could be called, so I did some digging into the source code and discovered that the info UseRouting() has to make available for both UseAuthentication() and UseAuthorization() is just the Endpoint class. Specifically, Endpoint.Metadata which is of type EndpointMetadataCollection.

EndpointMetadataCollection is just an object array, so to figure out what might actually be populated there, I just created an empty WebAPI project, set an authorization attribute above a controller, threw in some test middleware, and added a breakpoint right after assigning HttpContext.GetEndpoint().Metadata to a variable.

Turns out, one of the things it's populating is data about the Authorization attribute I added:

In hindsight, that makes a lot of sense. It would be dumb to try to figure out if a request was authorized before you even knew if the endpoint required authorization (or if a user was authenticated before we knew the request required authentication).

Something else I stumbled upon that was really insightful was this article by Areg Sarkissian that really gets into the nitty gritty of endpoint routing without being as dry as microsoft docs. This example in particular did an excellent job of showing what I mentioned above:

{
    if (env.IsDevelopment())
        app.UseDeveloperExceptionPage();
    else
        app.UseHsts();

    app.UseHttpsRedirection();

    app.UseRouting(routes =>
    {
        routes.MapControllers();

        //Mapped route that gets attached authorization metadata using the RequireAuthorization extension method.
        //This metadata will be added to the resolved endpoint for this route by the endpoint resolver
        //The app.UseAuthorization() middleware later in the pipeline will get the resolved endpoint
        //for the /secret route and use the authorization metadata attached to the endpoint
        routes.MapGet("/secret", context =>
        {
            return context.Response.WriteAsync("secret");
        }).RequireAuthorization(new AuthorizeAttribute(){ Roles = "admin" });
    });

    app.UseAuthentication();

    //the Authorization middleware check the resolved endpoint object
    //to see if it requires authorization. If it does as in the case of
    //the "/secret" route, then it will authorize the route, if it the user is in the admin role
    app.UseAuthorization();

    //the framework implicitly dispatches the endpoint at the end of the pipeline.
}

app.UseAddAuthentication() is a cross-cutting concern that occurs after routing is applied. Not all routes will have authentication applied to them.

Here's a suggested ordering:

// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
     app.UseMigrationsEndPoint();
}
else
{
    app.UseExceptionHandler("/Error");
    // The default HSTS value is 30 days. You may want to change this for 
 production scenarios, see https://aka.ms/aspnetcore-hsts.
     app.UseHsts();
}

 app.UseHttpsRedirection();
 app.UseStaticFiles();
 app.UseCookiePolicy();

 app.UseRouting();
 app.UseRequestLocalization();
 app.UseCors();

 app.UseAuthentication();
 app.UseAuthorization();
 app.UseSession();
 app.UseResponseCompression();
 app.UseResponseCaching();

 app.MapRazorPages();
 app.MapControllerRoute(
      name: "default",
     pattern: "{controller=Home}/{action=Index}/{id?}");

 app.Run();