Count number of threads used by Parallel.ForEach

user_145999 0
Viewed 4948

How can I determine the number of threads used during a specific call of Parallel.ForEach (or Parallel.Invoke, or Parallel.For)

I know how to limit the maximum number of threads, e.g.

Parallel.ForEach(myList, 
                 new ParallelOptions { MaxDegreeOfParallelism = 4 },
                 item => { doStuff(item); });

I know that the Task.Parallel library uses some heuristics to determine the optimal number of additional threadpool threads to use at runtime, in addition to the current thread; some value between 0 and MaxDegreeOfParallelism.

I would like to know how many threads have actually been used, for logging purposes:

Stopwatch watch = Stopwatch.StartNew();
Parallel.ForEach(myList, item => { doStuff(item); });
trace.TraceInformation("Task finished in {0}ms using {1} threads", 
       watch.ElapsedMilliseconds, NUM_THREADS_USED);

I mainly want this data logged for curiosity's sake, and to improve my understanding. It does not have to be 100% reliable, since I do not intend to use it for anything else.

Is there a way to get this number, without major performance penalties?

task-parallel-library .net-4.0 multithreading .net c#
4 Answers

You could use a (thread-safe) list to store the IDs of the used threads and count them:

ConcurrentBag<int> threadIDs = new ConcurrentBag<int>();
Parallel.ForEach(myList, item => { 
    threadIDs.Add(Thread.CurrentThread.ManagedThreadId);
    doStuff(item); 
});

int usedThreads = threadIDs.Distinct().Count();

This does have a performance impact (especially the thread-safety logic of ConcurrentBag), but I can't tell how big that is. The relative effect depends on how much work doStuff does itself. If that method has only a few commands, this thread counting solution may even change the number of used threads.

user_55285930

I know that the Task.Parallel library uses some heuristics to determine the optimal number of additional threadpool threads to use at runtime, in addition to the current thread; some value between 0 and MaxDegreeOfParallelism. I would like to know how many threads have actually been used, for logging purposes

Since you mention the thread pool and MaxDoP, I interpreted this question as you wanted to know how many concurrent threads were used at any one time. This you can find out by using a field and Interlocked.

class MyClass
{
  private int _concurrentThreadCount;
  private ILog _logger; //for example
  
  public void DoWork()
  {
    var listOfSomething = GetListOfStuff();
    Parallel.ForEach(listOfSomething, singleSomething => 
    {
        Interlocked.Increment(ref _concurrentThreadCount);
        _logger.Info($"Doing some work. Concurrent thread count: {_concurrentThreadCount}");
        
        // do work
        
        Interlocked.Decrement(ref _concurrentThreadCount);
    });
  }
}
Andrew_163910

In your DoStuff method you can add the code like this

    private void DoStuff(T item)
        {
            Logger.Log($"Item {item.ToString()} was handled by thread # {Thread.CurrentThread.ManagedThreadId}");
            // your logic here
        }
user_20395220

While I am aware this is an older question, I followed up on Evk's suggestion. Also not sure about the performance impact, but you could use a concurrentdictionary to keep track of the threadids:

var threadIDs = new ConcurrentDictionary<int, int>();
Parallel.ForEach(myList, item => { 
    threadIDs.TryAdd(Thread.CurrentThread.ManagedThreadId, 0);
    doStuff(item); 
});

int usedThreads = threadIDs.Keys.Count();
user_7749980
Related