Why do they prefer dynamic scheduling in this code snippet

user_4113654 0
Viewed 111

I solved an exercise, where I had to implement SGD (stochastic gradient descent) with momentum. The exercise was to parallelize it afterwards.

My suggestion was as follow:

#pragma omp for schedule(static) nowait
for (int i = 0; i < size; i++)
    {
         const double nablaE_w = (1.0/double(B)) * (grad[i] + lambda * param[i]);
         mom_w[i] = beta * mom_w[i] - eta * nablaE_w;
         param[i] = param[i] + mom_w[i];
    }

I use nowait because after the for-loop the calculation is finished.

But on the solution they uses:

#pragma omp for schedule (dynamic, 64/sizeof(double)) nowait

Also after reading several stackoverflow-answers, I still cannot see the advantage of scheduling(dynamic) and why they uses the chunksize=8

I am careful when using schedule(dynamic) - actually I never use it, because I have no idea how to do it correctly.

openmp c++11 c++
1 Answers

The solution likely did it for one of two reasons: there is another nowait loop nearby that is imbalanced and might create a case in which threads reach this loop out of order, or they were worried about running on a system with a great deal of interference that is variable by core. Using schedule(dynamic) is materially more expensive, for now until runtimes actually implement nonmonotonic properly, so static is the clear choice as long as the workload is balanced, which this snippet is. The only reason to use dynamic here would be if imbalance between threads were expected to come from somewhere else, either in the code or from interfering workloads.

user_24865270
Related