UseExceptionHandler won't catch tasks exceptions?

To answer your questions.

Why does the task exception isn't recognized by UseExceptionHandler?

As already suggested in the comments, you cannot use UseExceptionHandler to catch exceptions initiated inside non-awaited tasks. UseExceptionHandler wraps your request in ASP.NET Core middleware. Once the action returns OK to the client, the middleware is no longer able to catch any exceptions happening in tasks started from within the action.

How can I catch ALL types of exceptions? Should I rely only on AppDomain.CurrentDomain.FirstChanceException?

You can catch exceptions globally and log them this way if you'd like. But I wouldn't recommend you to do it this way. The only reason you need to implement this event, is that you are starting tasks/threads inside your web requests. You have no way of knowing if these tasks are kept running (application restart, recycle, etc.). If you are looking to launch background tasks with ASP.NET Core, you should use Worker Services which is the intended way of doing this:

.ConfigureServices((hostContext, services) =>
{
    services.AddHostedService<MyWorker>();
});

public class MyWorker : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            try
            {
                // Do work
            }
            catch (Exception e)
            {
                // Log it?
            }

            await Task.Delay(TimeSpan.FromMinutes(5), stoppingToken);
        }
    }
}

The cause of this particular symptom is that Get is starting a fire-and-forget task that the server knows nothing about. The request will complete before the task even has a chance to execute, so the UseExceptionHandler middleware will never see any exceptions. This is no longer a fire-and-forget task.

The real problem though, is executing a long running task in the background. The built-in way to do this is using a Background Service. The docs show how to create timed and queued background service, that act as job queues.

It's equally easy, if not easier, to publish messages with the desired data from, eg a controller to the background service using, eg Channels. No need to create our own queue, when the BCL already has an asynchronous one.

The service could look like this :

public class MyService: BackgroundService 
{

    private readonly ChannelReader<T> _reader;

    public QueuedBspService(MessageQueue<T> queue)
    {
        _reader = queue.Reader;
    }

    protected internal async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        try
        {
            await foreach (var msg in _reader.ReadAllAsync(stoppingToken))
            {                    

                try
                {
                    //Process the message here
                }
                catch (Exception exc)
                {
                    //Handle message-specific errors
                }
            }
        }
        catch (Exception exc)
        {
            //Handle cancellations and other critical errors
        }
    }
}

The MessageQueue<T> wraps the Channel, making it easier to inject it to both the BackgroundService and any publishers like eg, a Controller action:

public class MessageQueue<T> 
{
    private readonly Channel<T> _channel;

    public ChannelReader<T> Reader => _channel;
    public ChannelWriter<T> Writer => _channel;

    public MessageChannel()
    {
        _channel = Channel.CreateBounded<T>(1);
    }
}

I adjusted this code from a service that only allows a single operation at a time. That's a quick&dirty way of preventing controllers from making requests that can't be handled.

On the contolle side, this action will post a request to the queue if possible, and return a Busy response otherwise :

public class MyController
{
    private readonly ChannelWriter<T> _writer;


    public MyController(MessaggeQueue<T> queue)
    {
        _writer = queue.Writer;
    }

    [HttpPost]
    [ProducesResponseType(StatusCodes.Status201Created)]
    [ProducesResponseType(StatusCodes.Status503ServiceUnavailable)]
    public async Task<ActionResult> Post(....)
    {
        var jobName="SomeJob";
        var id=Guid.NewGuid();
        var jobMsg=CreateMessage(id,...);


        try
        {
            if (_writer.TryWrite(msg))
            {
                return CreatedAtAction("GetItem","Jobs",new {id});
            }
            else
            {
                return Problem(statusCode:(int) HttpStatusCode.ServiceUnavailable,detail:"Jobs in progress",title:"Busy");
            }
        }
        catch (Exception exc)
        {
            _logger.LogError(exc,"Queueing {job} failed",jobName);
            throw;
        }
    }            
}

The Post action first checks if it can even post a job message. If it succeeds, it returns a 201 - Created response with a URL that could be checked eg to check the status of the jobs. return Created() could be used instead, but once you create a long running job, you also want to check its status.

If the channel is at capacity, the core returns 503 with an explanation