I often end up with several nested foreach loops and sometimes when writing general functions (e.g. for a package) there is no level which is obvious to parallelize at. Is there any way to accomplish what the mock-up below describes?
foreach(i = 1:I) %if(I < J) `do` else `dopar`% {
foreach(j = 1:J) %if(I >= J) `do` else `dopar`% {
# Do stuff
}
}
Furthermore, is there some way to detect if a parallel backend is registered so I can avoid getting unnecessary warning messages? This would be useful both when checking packages prior to CRAN submission and to not bother users running R on single core computers.
foreach(i=1:I) %if(is.parallel.backend.registered()) `dopar` else `do`% {
# Do stuff
}
Thanks for your time.
Edit: Thank you very much for all the feedback on cores and workers and you're right in that the best way to deal with the above example would be to rethink the whole setup. I'd prefer something like to below to the triu idea but it's essentially the same point. And it could of course also be done with a parallel tapply like Joris suggested.
ij <- expand.grid(i=1:I, j=1:J)
foreach(i=ij$I, j=ij$J) %dopar% {
myFuction(i, j)
}
However, in my attempt to simplify the situation that gave rise to this thread I left out some crucial details. Imagine that I have two functions analyse and batch.analyse and the best level to parallelize at might be different depending on the values of n.replicates and n.time.points.
analyse <- function(x, y, n.replicates=1000){
foreach(r = 1:n.replicates) %do% {
# Do stuff with x and y
}
}
batch.analyse <- function(x, y, n.replicates=10, n.time.points=1000){
foreach(tp = 1:time.points) %do% {
my.y <- my.func(y, tp)
analyse(x, my.y, n.replicates)
}
}
If n.time.points > n.replicates it makes sense to parallelize in batch.analyse but otherwise it makes more sense to parallelize in analyse. Any ideas on how to tackle it? Would it somehow be possible to detect in analyse if parallelization has already taken place?