R - parallelisation error, checkCluster(cl) - not a valid cluster

user_8588748 0
Viewed 4284

This code brings me an error: Error in checkCluster(cl): not a valid cluster

library(parallel)
numWorkers <-8

cl <-makeCluster(numWorkers, type="PSOCK")

   res.mat <- parLapply(1:10, function(x) my.fun(x))

stopCluster(cl)

Without parallelisation attempts this works totally fine:

res.mat <- lapply(1:10, function(x) my.fun(x))

And this example works very well too:

workerFunc <- function(n){return(n^2)}

library(parallel)

numWorkers <-8

cl <-makeCluster(numWorkers, type ="PSOCK")

res <- parLapply(cl, 1:100, workerFunc)

stopCluster(cl)

print(unlist(res))

How can i solve my problem?

I found for example 

class(cl)
[1] "SOCKcluster" "cluster"

an cl is:

cl
socket cluster with 8 nodes on host ‘localhost’
parallel-processing r
2 Answers
library(parallel)
numWorkers <- 8

cl <-makeCluster(numWorkers, type="PSOCK")

res.mat <- parLapply(cl,1:10, function(x) my.fun(x))

stopCluster(cl)
user_133102970

Just to be excessively specific, the problem with res.mat <- parLapply(1:10, function(x) my.fun(x)) is not necessarily the order of the arguments, but that the argument cl is not specified.

cl <-makeCluster(numWorkers, type ="PSOCK")
res.mat <- parLapply(x = 1:10,
                     fun = function(x) my.fun(x),
                     cl = cl
                     )

should work, because all required arguments are specified. Alternatively, ?parLapply indicates that parLapply uses the default cluster if cl is not specified. A default cluster can be set using parallel::setDefaultCluster(), which then allows parLapply to revert to default behaviour when cl is not included in the user input.

cl <-makeCluster(numWorkers, type ="PSOCK")
parallel::setDefaultCluster(cl)
res.mat <- parallel::parLapply(x = 1:10,#by default cl = NULL if not specified
                               fun = function(x) my.fun(x),
                               )
user_37453530
Related