Why does this for loop not work in parallel?

user_5071733 0
Viewed 92

I am trying to run a function in parallel in a linux version of R. However, I keep finding that each parallel iteration returns NULL rather than the output of the function. I have isolated the problem to a for loop.

Here is a simplified version:

library(parallel)

rows <- seq(1,9,1)

for_test <- function(i){
  for (s in 1:3){
    print(i*s) 
  }
}

cls <- makeCluster(length(rows), type = 'FORK')

parLapply(cls, rows, for_test)

stopCluster(cls)

This returns NULL for each of the nine iterations, but changing the for_test function to:

for_test <- function(i){
  s <- 2
  print(i*s)
}

returns the result of i*s rather than NULL. I have run for loops within parallel previously and can't seem to figure out why it isn't working on this occasion. Is anyone able to point to where I am going wrong?

parallel-processing r
1 Answers

Firstly: the loop works in parallel, we just don't see the print. The returned NULL is not a result of the print function, it is the list of function return values from parallel calls. Instead of print, collect the values and return it. Printing to an external file will also work, but I suggest to start with the ordinary way first, as parLapply collates the return values in a convenient way.

As an example how to use return values, try the following:

library(parallel)

rows <- seq(1,9,1)

for_test <- function(i){
  txt <- NULL
  for (s in 1:3){
    txt <- rbind(txt, c(i, s, i*s))
  }
  txt
}

cls <- makeCluster(length(rows))
parLapply(cls, rows, for_test)
stopCluster(cls)

Explanation:

In the first example of the OP printis within a for loop, while in the second version it is the last statement. print returns a value while for()returns NULL.

Demo:

> x <- print(2)
[1] 2
> x
[1] 2
> 
> x <- for (i in 1:2) print(2 * i)
[1] 2
[1] 4
> x
NULL
>
user_36775760
Related