"pragma omp parallel for" in function not effective when function is called within another parallel loop

user_10209840 0
Viewed 907

Inside a parallelized 'for' loop I'm calling a function, which in turn contains an expensive calculation that I would like to parallelize with the remaining CPUs. However, my inner 'parallel for' directive is simply ignored.

This is for an image processing pipeline. The camera has 4 detectors and takes e.g. 100 images. I want to calculate for each detector the median of the 100 input images. Hence my outer loop uses 4 threads for the 4 detectors, and I want to use the remaining CPUs to parallelize the median calculation in the inner loop.

int main()
{
  // OUTER LOOP over various detectors
  # pragma omp parallel for
  for (int det=0; det<4; ++det) {
      // some serial work
      calculate_median(det);
  }
}

void calculate_median(int det)
{
   // some serial work

   #pragma omp parallel for num_threads(available_additional_threads)
   for (int pixel =0; pixel < numpixels; ++pixel) {
        // create a stack of pixels from the 100 input images
        // calculate median of stack
   }

   // more serial work
}

I calculate the the additional available threads based on how many detectors the camera has and on the number of CPUs on the user's machine. Like this there are never more threads running than there are CPUs.

What I observe is that the inner 'omp parallel for' is ignored, i.e. no matter the value of 'available_additional_threads', I get only a single thread. HOWEVER, if I do set the outer loop to just a single thread, then the inner parallelization works as expected. This really puzzles me. Thank you very much for any additional insight!

UPDATE / SOLUTION: Setting 'omp_set_nested(true)' enabled the inner loop parallelization.

openmp multithreading nested c++
1 Answers

You can use the collapse clause the compiler supports OpenMP 3.0

#pragma omp parallel for collapse(2)
for (int x = 0; x < x_max; ++x) {
    for (int y = 0; y < y_max; ++y) { 
    //parallelize this code here
    }
//IMPORTANT: no code in here
}

,but this seams to be not applicable here, since you have to do some serial work.

My experience is, that nested loops have a worse performance than one parallel loop, which is nicely panellized.

Usually, it is a good idea to just parallelize the outer loop. Since your outer loop only has 4 iterations, I think it is reasonable to use OMP just for the inner loop. The scheduler can handle load imbalances her better.

Keep in mind the golden rule of performace refactorings: Measure.

I suggest, that you use a tool such as Intel's Vtune Ampifier to measure how your performance is and if a nested loop would really help you.

user_24166470
Related