Many connections in TIME_WAIT state in connection pool, using either static or single HttpClient instance in WebApi

I've been strugling for days trying to prevent multiple connections in pool when using HttpClient

I've built a REST WebApi from which I'm calling an external endpoint by using HttpClient inside of it. A controller's action is called multiple times by a client in order to perform aforementioned external call.

Previously I was using WebClient but it suffered of SocketExhaustation when several requests were made. Searching on the web I found that migrating to HttpWebClient will solve my issues if used statically or within a singleton. I even tried to implement IHttpClientFactory but it was useless, as I experienced the same issue on my injected services.

This is my current implementation, after several refactors. Sorry if currently this code is not optimal but after multiple rewritings this was my last resort.

public static class CustomHttpClientManager
    {
        private static HttpClient _httpClient;

        public static void Initialize()
        {
            ServicePointManager.SecurityProtocol = SecurityProtocolType.Tls12 | SecurityProtocolType.Tls11 | SecurityProtocolType.Tls;

            ServicePointManager.ServerCertificateValidationCallback +=
                    (sender, cert, chain, sslPolicyErrors) => true;

            CookieContainer cookies = new CookieContainer();
            _httpClient = new HttpClient(
            new SocketsHttpHandler()
            {
                Proxy = new CustomProxy(),
                UseProxy = true,
                PooledConnectionLifetime = TimeSpan.FromSeconds(30),
                PooledConnectionIdleTimeout = TimeSpan.FromSeconds(20),
                MaxConnectionsPerServer = 2,
                CookieContainer = cookies,
                AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate
            });

            _httpClient.DefaultRequestHeaders.ConnectionClose = false;
        }

        public static async Task<string> GetAsString(string url){
            HttpResponseMessage response = null;
            try{

                HttpRequestMessage httpRequestMessage = new HttpRequestMessage()
                {
                    RequestUri = new Uri(url),
                    Method = HttpMethod.Get
                };

                httpRequestMessage.Headers.Add("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
                httpRequestMessage.Headers.Add("Accept-Encoding", "gzip,deflate,sdch");
                httpRequestMessage.Headers.Add("Accept-Language", "en-GB,en-US;q=0.8,en;q=0.6");
                httpRequestMessage.Headers.Add("Accept-Charset", "ISO-8859-1,utf-8;q=0.7,*;q=0.3");
                
                response = await _httpClient.SendAsync(httpRequestMessage);    
                if(response.IsSuccessStatusCode){
                    return await response.Content.ReadAsStringAsync();   
                }
                else{
                    throw new Exception(ex.Message);
                }
            }
            catch(Exception ex){
                return $"Could not fetch response from: {connectionParameters.Url}, due to: {ex.Message}";     
            }
            finally{
                if(response != null){
                    response.Dispose();
                }
            }
        }
    }

The callstack until reaching the current code goes like this:

1. At Startup the WebApi app calls CustomHttpClientManager's Initialize method, which instantiates the HttpClient only once.
2. A client calls WebApi controller's action.
3. The action calls a business logic class (service class)
4. The logic class uses the CustomHttpClientManager's GetAsString method to perform the external call by using previously instantiated HttpClient

The client performs a lot of calls to the WebApi (around 200 rq per minute), and when running netstat -ano on the command prompt, it shows a quite long list of pooled connections whose status is TIME_WAIT, which shows that HttpClient is not reusing connections from the pool at all. These connections only dissappear around 2 minutes after being used.

Besides this code my other approaches were.

• Injecting IHttpClientFactory on Startup by using .AddHttpClient()
• Using the same class above, but using a thread safe singleton version.

None of them worked. I don't know what else to do. This has driven me crazy for around 2 days straight.

I've tested this implementation from a simple console app, and the issue is not happening. This suggests me that somehow WebApi action keeps closing the external connection once a reply to its caller has been given, and once another call is made, it creates another connection in the pool and it goes on and on for the next incomming requests as well.

BTW, I'm using .NET Core 3.1

Thanks in advance