How to scale an application with 50000 Simultaneous Tasks

user_10963245 0
Viewed 330

I am working on a project which needs to be able to run (for example) 50,000 tasks simultaneously. Each task will run at some frequency (say 5 minutes) and will be either a url ping or an HTTP GET request. My initial plan was to create thread for each task. I ran a basic test to see if this was possible given available system resources. I ran the following code as a console app:

public class Program
{
    public static void Test1()
    {
        Thread.Sleep(1000000);
    }

    public static void Main(string[] args)
    {
        for(int i = 0; i < 50000; i++)
        {
            Thread t = new Thread(new ThreadStart(Test1));
            t.Start();
            Console.WriteLine(i);
        }
    }
}

Unfortunately, though it started very fast, at the 2000 thread mark, the performance was greatly decreased. By 5000, I could count faster than the program could create threads. This makes getting to 50000 seem like it wouldn't be exactly possible. Am I on the right track or should I try something else? Thanks

multithreading c#
1 Answers

Many people have the idea that you need to spawn n threads if you want to handle n tasks in parallel. Most of the time a computer is waiting, it is waiting on I/O such as network traffic, disk access, memory transfer for GPU compute, hardware device to complete an operation, etc.

Given this insight, we can see that a viable solution to handling as many tasks in parallel as possible for a given hardware platform is to pipeline work: place work in a queue and process it using as many threads as possible. Usually, this means 1-2 threads per virtual processor.

In C# we can accomplish this with the Task Parallel Library (TPL):

    class Program
    {
        static Task RunAsync(int x)
        {
            return Task.Delay(10000);
        }

        static async Task Main(string[] args)
        {
            var tasks = Enumerable.Range(0, 50000).Select(x => RunAsync());

            Console.WriteLine("Waiting for tasks to complete...");

            await Task.WhenAll(tasks);

            Console.WriteLine("Done");
        }
    }

This queues 50000 work items, and waits until all 50000 tasks are complete. These tasks only execute on as many threads that are needed. Behind the scenes, a task scheduler examines the pool of work and has threads steal work from the queue when they need a task to execute.

Additional Considerations

With a large upper bound (n=50000) you should be cognizant of memory pressure, garbage collector activity, and other task-related overhead. You should consider the following:

  • Consider using ValueTask<T> to minimize allocations, especially for synchronous operations
  • Use ConfigureAwait(false) where possible to reduce context switching
  • Use CancellationTokenSource and CancellationToken to cancel requests early (e.g. timeout)
  • Follow best practices
    • Avoid awaiting inside of a loop where possible
    • Avoid querying tasks too frequently for completion
    • Avoid accessing Task<T>.Result before a task is complete to prevent blocking
    • Avoid deadlocks by using synchronization primitives (mutex, semaphore, condition signal, synclock, etc) as appropriate
    • Avoid frequent use of Task.Run to create tasks to avoid exhausting the thread pool available to the default task scheduler (this method is usually reserved for compute-bound tasks)
user_3817080
Related