Memory leak in httpClient.GetAsync C# .NET 4.8

Question

I have a memory leak issue when using httpClient.GetAsync.
In the task manager, the processes stay with a high memory consumption that doesn't get freed.

This is my code:

HttpClient httpClient = new HttpClient();
httpClient.Timeout = TimeSpan.FromMinutes(2);

try
{
    using (var response = await httpClient.GetAsync(fullURL))
    {
        if (!response.IsSuccessStatusCode)
        {
            Logger.Default.Error($"error code {(int)response.StatusCode} - {response.StatusCode}");
            return null;
        }
        using (MemoryStream memStream = new MemoryStream())
        {
            await response.Content.CopyToAsync(memStream);
            Logger.Default.Debug($"finished reading response, sized {Math.Round(memStream.Length / Math.Pow(1024, 2), 2)} MB");
        }
    }
}
catch (TaskCanceledException ex)
{
    Logger.Default.Error($"Request timed out. {ex.Message}
{ex.StackTrace}");
    return null;
}

However, when I use httpClient.GetStreamAsync instead, and change:

await response.CopyToAsync(memStream);

To:

await response.Content.CopyToAsync(memStream);

The memory does get released after a few seconds. But I rather use GetAsync since it provides me information about the status code, which GetStreamAsync doesn't.

I've already tried calling the Garbage collector (GC.Collect(2)) after the object got disposed, but it didn't help.

What am I doing wrong?

user_9974025 · Answer

HTTPClient has given me issues like this before. I found this article very helpful:

https://www.aspnetmonsters.com/2016/08/2016-08-27-httpclientwrong/

Basically, even though it implements IDisposable, you can instead instantiate it for the lifetime of the application as a singleton. This allows the app to only use that HTTPClient connection, and won't lead to issues where the connection is not disposed properly.

If you use it within a using block, Windows will hold a connection in this state for 240 seconds. It is set by:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\TcpTimedWaitDelay]

Try this out:

using System;
using System.Net.Http;

namespace ConsoleApplication
{
    public class Program
    {
        private static HttpClient Client = new HttpClient();
        public static async Task Main(string[] args) 
        {
            Console.WriteLine("Starting connections");
            for(int i = 0; i<10; i++)
            {
                var result = await Client.GetAsync("http://aspnetmonsters.com");
                Console.WriteLine(result.StatusCode);
            }
            Console.WriteLine("Connections done");
            Console.ReadLine();
        }
    }
}

If you implement IDisposable by the using statement syntax, the connection actually stays open after the end of the using block. Then, when your application comes across another using block of HTTPClient, the previously existing once is not closed yet. This can lead to all sorts of errors, most commonly:

There is a limit to how quickly Windows can open new sockets so if you exhaust the connection pool then you’re likely to see an error like:

Unable to connect to the remote server
System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted.

Some others on the internet have found better ways to do this, I don't personally have experience with them, but here are some links worth checking out. These describe ways to use IHTTPClientFactory and other ways to do this without establishing an HTTPClient Singleton:

https://josef.codes/you-are-probably-still-using-httpclient-wrong-and-it-is-destabilizing-your-software/

https://www.stevejgordon.co.uk/httpclient-creation-and-disposal-internals-should-i-dispose-of-httpclient

Memory leak in httpClient.GetAsync C# .NET 4.8

1 Answers