Attaching middleware to a specific route in ASP.NET Core Web API?

Inside the configure, I can attach a global middleware using:

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
   ....
   app.UseMiddleware<MyMiddleware>();
   ...
}

This will apply to all actions.

However, I thought to myself, how can I attach a middleware to a specific route/action? (Sure I can put some if's inside the code but I don't like the approach)

But then I saw this:

 app.UseEndpoints(endpoints =>
  {
      endpoints.Map("/version", endpoints.CreateApplicationBuilder()
               .UseMiddleware<MyMiddleware>()
               .UseMiddleware<VersionMiddleware>()
               .Build())
               .WithDisplayName("Version number"); 
  }

This will work but will create a NEW endpoint /version.

Question

How can I attach custom middleware to an existing controller action route?

I've tried:

endpoints.Map("/weatherforecast", endpoints.CreateApplicationBuilder()
    .UseMiddleware<MyMiddleware>()
    .UseMiddleware<VersionMiddleware>()
    .Build())
    .WithDisplayName("Version number");

But it doesn't seem to affect. I see a regular response from the controller. Without new headers which the middleware adds.