Parallel processing of independent functions with multiple arguments in R

user_7764533 0
Viewed 60

I have the following function which is a much simpler version of the functions that I have but it pictures what I'm struggling to do.

I follow the discussion here Parallelize a function in R (not a loop!) which is a solution for a single argument for each function.

In my case I have the following function unparal which has multiple arguments that can be shared across the functions f_1,f_2,f_3,f_4


f_1 = function(x,y,z){
  return(rnorm(z,x,y))
}
f_2 = function(x,y,z){
  return(rgamma(z,x,y))
}
f_3 = function(x,y,z){
  return(rbeta(z,x,y))
}
f_4 = function(x,y,z){
  return(runif(z,x,y))
}
unparal = function(x,g,t,y,z){
  res1 = f_1(x,g,z)
  res2 = f_2(g,t,z)
  res3 = f_3(t,y,z)
  res4 = f_4(x,y,z)
  res = c(res1,res2,res3,res4)
  return(res)
}

What I would like to do is to parallelize the function unparal, i.e. to run all the functions f_1,f_2,f_3,f_4 in parallel. Is there a convenient way to do that?

parallel-processing r
1 Answers

There's a bunch of ways. One is to use the future package. E.g. like so:

library(future)

unparal = function(x,g,t,y,z){
  require(future)
  
  res1 %<-% f_1(x,g,z) %seed% TRUE
  res2 %<-% f_2(g,t,z) %seed% TRUE
  res3 %<-% f_3(t,y,z) %seed% TRUE
  res4 %<-% f_4(x,y,z) %seed% TRUE
  res = c(res1,res2,res3,res4)
  return(res)
}

plan(multisession, workers = 4)
unparal(1, 2, 3, 4, 5)
plan(sequential)

The %<-% operator does an assignment, but this is done according to the current plan, so in this case in 1 of the 4 workers. Then when the result is accessed (in res = c(res1,res2,res3,res4)), the main process will wait till all four results have been returned.

Note the %seed% TRUE part is only necessary when you need correct random number generation.

user_43414400
Related