Concurrent requests with HttpClient take longer than expected

Question

I have a webservice which receives multiple requests at the same time. For each request, I need to call another webservice (authentication things). The problem is, if multiple (>20) requests happen at the same time, the response time suddenly gets a lot worse.

I made a sample to demonstrate the problem:

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Net;
using System.Net.Http;
using System.Threading.Tasks;

namespace CallTest
{
    public class Program
    {
        private static readonly HttpClient _httpClient = new HttpClient(new HttpClientHandler { Proxy = null, UseProxy = false });

        static void Main(string[] args)
        {
            ServicePointManager.DefaultConnectionLimit = 100;
            ServicePointManager.Expect100Continue = false;

            // warmup
            CallSomeWebsite().GetAwaiter().GetResult();
            CallSomeWebsite().GetAwaiter().GetResult();

            RunSequentiell().GetAwaiter().GetResult();

            RunParallel().GetAwaiter().GetResult();
        }

        private static async Task RunParallel()
        {
            var tasks = new List();
            for (var i = 0; i < 300; i++)
            {
                tasks.Add(CallSomeWebsite());
            }
            await Task.WhenAll(tasks);
        }

        private static async Task RunSequentiell()
        {
            var tasks = new List();
            for (var i = 0; i < 300; i++)
            {
                await CallSomeWebsite();
            }
        }

        private static async Task CallSomeWebsite()
        {
            var watch = Stopwatch.StartNew();
            using (var result = await _httpClient.GetAsync("http://example.com").ConfigureAwait(false))
            {
                // more work here, like checking success etc.
                Console.WriteLine(watch.ElapsedMilliseconds);
            }
        }
    }
}

Sequential calls are no problem. They take a few milliseconds to finish and the response time is mostly the same.

However, parallel request start taking longer and longer the more requests are being sent. Sometimes it takes even a few seconds. I tested it on .NET Framework 4.6.1 and on .NET Core 2.0 with the same results.

What is even stranger: I traced the HTTP requests with WireShark and they always take around the same time. But the sample program reports much higher values for parallel requests than WireShark.

How can I get the same performance for parallel requests? Is this a thread pool issue?

user_2829009 · Answer

This behaviour has been fixed with .NET Core 2.1. I think the problem was the underlying windows WinHTTP handler, which was used by the HttpClient.

In .NET Core 2.1, they rewrote the HttpClientHandler (see https://blogs.msdn.microsoft.com/dotnet/2018/04/18/performance-improvements-in-net-core-2-1/#user-content-networking):

In .NET Core 2.1, HttpClientHandler has a new default implementation implemented from scratch entirely in C# on top of the other System.Net libraries, e.g. System.Net.Sockets, System.Net.Security, etc. Not only does this address the aforementioned behavioral issues, it provides a significant boost in performance (the implementation is also exposed publicly as SocketsHttpHandler, which can be used directly instead of via HttpClientHandler in order to configure SocketsHttpHandler-specific properties).

This turned out to remove the bottlenecks mentioned in the question.

On .NET Core 2.0, I get the following numbers (in milliseconds):

Fetching URL 500 times...
Sequentiell   Total: 4209, Max:  35, Min: 6, Avg:  8.418
Parallel      Total:  822, Max: 338, Min: 7, Avg: 69.126

But on .NET Core 2.1, the individual parallel HTTP requests seem to have improved a lot:

Fetching URL 500 times...
Sequentiell   Total: 4020, Max:  40, Min: 6, Avg:  8.040
Parallel      Total:  795, Max:  76, Min: 5, Avg:  7.972

Concurrent requests with HttpClient take longer than expected

4 Answers