custom error middleware in net core not catching model state errors

I have some custom middleware set up to handle errors in my api application.

in startup.cs i have got code as follows

app.UseErrorHandlingMiddleware(); //this is custom extension method that just does builder.UseMiddleware<ErrorHandlingMiddleware>();
app.UseAuthentication;
app.UseMvc();

any errors thrown in the application are caught by the middleware but if i send invalid model to a controller action the model state validation error is not getting caught by my middleware. I get a standard 400 bad request response such as:

{"errors":{"Username":["The Username field is required."]},"title":"One or more validation errors occurred.","status":400,"traceId":"0HLNLAV1FS4IJ:00000002"}

Is there something i need to do to have my middleware get these errors?

The middleware invoke methods is as follows:

 public async Task Invoke(HttpContext context /* other dependencies */)
    {
        try
        {
            await next(context);
        }
        catch (Exception ex)
        {
            await HandleExceptionAsync(context, ex);
        }
    }

Do i need to use an ActionFilter to check modelstate and throw exception if error in order for my error middleware to get it?