I'm writing a raytracer with EM-processing (radar application). The following is done for many different simulation locations of the radar:
- raytracing, generates a lot of data
- EM-postprocessing, generates a scalar and then deletes the raytrace data
Each simulation point will be encapsulated in its own instance of a class (all grouped in a std::vector) with a specification of the radar location for that simulation point, references to data it will only read (shared by all simulation points), and properties for storing its results (so each simulation point has its own). Because of this setup I thought I could benefit of using a for_each loop with std::execution::par_unseq policy without taking further measures. Correct?
The problem however is that the raytracing generates so much data that, when for example having 10 thousand simulation locations, I may run out of memory when the scheduler decides to do all the raytracing first because it is allowed to do so with par_unseq. So my idea was to write a normal for loop which feeds an inner parallel for_each loop with, say, 100 simulation points at the time. Is this an optimal solution for my case? Or did I totally misinterpret how parallel things work?