Authorized get request not working in UWP

Im creating a UWP application that gets data from an ASP.NET API When im trying to make a request that requires a JWT token it fails.

the request code:

public async Task<string> testAsync(string token)
        {
            token = token.Replace("\"", "");

            var authenticationHeaderValue = new AuthenticationHeaderValue("Bearer", token);
            client.DefaultRequestHeaders.Authorization = authenticationHeaderValue;

            var json = await client.GetAsync("http://localhost:5000/api/TravelList");
            return json.Content.ReadAsStringAsync().Result;
        }

client is a HttpClient object created like this:

client = new HttpClient(new HttpClientHandler
            {
                ServerCertificateCustomValidationCallback = (a, b, c, d) => true
            });

When I run this code I get: System.Net.HttpStatusCode.InternalServerError

And my backend logs this:

An unhandled exception has occurred while executing the request. System.InvalidOperationException: No authenticationScheme was specified, and there was no DefaultChallengeScheme found.

But when I test this in on swagger everything works fine: swagger

Note: I am using HTTP instead of HTTPS in my UWP application so I am using http://localhost:5000/... instead of https://localhost:5001/... but also this does work on swagger.

What am I doing wrong?