specify which API is documented by Swagger

You can put an ApiExplorerSettings attribute on a controller to remove it from Swagger:

[ApiExplorerSettings(IgnoreApi = true)]
public class TestApiController : ControllerBase
{
}

If you want to apply this on a lot of controllers based on some logic, it can be done e.g. with an action model convention: https://github.com/juunas11/AspNetCoreHideRoutesFromSwagger/blob/983bad788755b4a81d2cce30f82bc28887b61924/HideRoutesFromSwagger/Controllers/SecondController.cs#L18-L28

public class ActionHidingConvention : IActionModelConvention
{
    public void Apply(ActionModel action)
    {
        // Replace with any logic you want
        if (action.Controller.ControllerName == "Second")
        {
            action.ApiExplorer.IsVisible = false;
        }
    }
}

The convention is added in ConfigureServices like:

services.AddControllers(o =>
{
    o.Conventions.Add(new ActionHidingConvention());
});