Identity Server Bearer was not authenticated

I have a dockerised .NET Core project running Identity Server 4, which is correctly creating accounts, giving out tokens, and authenticating requests within its own container. I'll call this my AuthAPI

My problem is when I try to use my AuthAPI to validate tokens on other APIs

I have an API I'll call my CustomerAPI. I'm trying to use the Identity Server on the AuthAPI to Authorize my endpoints, but so far I've had no luck

The way I'm testing this functionality is by the following steps:

1. Create an Account on the AuthAPI
2. Confirm the AuthAPI Account email
3. Authenticate on the AuthAPI to retrieve my Bearer Token
4. Make an request to the AuthAPI to get the Account ID to use in my CustomerAPI endpoint route
5. Put this Bearer token in an Authorization Header and make a request to a CustomerAPI endpoint with the Authorize attribute

My authorized call to the AuthAPI using my Bearer Token authenticates and returns successfully. I can see in my Identity Server debug logs the following phrase:

AuthenticationScheme: Bearer was successfully authenticated.

When I attempt to use that same Bearer Token for my CustomerAPI endpoint I get the following debug output:

AuthenticationScheme: Bearer was not authenticated.

I assume this is due to my CustomerAPI configuration failure but haven't yet managed to find what the problem is, here is my configuration:

            services.AddAuthentication(options =>
            {
                options.DefaultAuthenticateScheme = JwtBearerDefaults.AuthenticationScheme;
                options.DefaultChallengeScheme = JwtBearerDefaults.AuthenticationScheme;                
            })
            .AddJwtBearer(o =>
            {
                o.Authority = "http://authapi/";
                o.Audience = "http://authapi/";
                o.RequireHttpsMetadata = false;
            });

Any help or guidance would be much appreciated, thanks!

--- Edit --- Requested Config file:

    public class Config
    {
        public static IEnumerable<IdentityResource> GetIdentityResource()
        {
            return new List<IdentityResource>
            {
                new IdentityResources.OpenId(),
                new IdentityResources.Email(),
                new IdentityResources.Profile()
            };
        }

        public static IEnumerable<ApiResource> GetApiResources()
        {
            return new List<ApiResource>
            {
                new ApiResource("customerapi", "Customer API")
                {
                    Scopes = {new Scope("api.read")}
                }
            };
        }

        public static IEnumerable<Client> GetClients()
        {
            return new []
            {
                new Client
                {
                    RequireConsent = false,
                    ClientId = "demo_site",
                    ClientName = "Demo Site",
                    AllowedGrantTypes = GrantTypes.Implicit,
                    AllowedScopes = {"openid", "email", "profile", "api.read"},
                    RedirectUris = {"http://localhost:4200/auth-callback"},
                    PostLogoutRedirectUris = {"http://localhost:4200/"},
                    AllowedCorsOrigins = {"http://localhost:4200"},
                    AllowAccessTokensViaBrowser = true,
                    AccessTokenLifetime = 3600
                }
            };
        }
    }