foreach memory usage in R

Question

I am trying the following code which includes a foreach loop to compute the normalized columns of a matrix A:

library(doParallel)
library(tictoc)

A <- matrix(1.0, 5000, 1000)

cl <- makeCluster(2)
registerDoParallel(cl)
gcinfo(TRUE)
tic()
res1 <- foreach(i=1:nrow(A), .combine='rbind') %dopar% (A[i,]/mean(A[i,])) 
toc()
gcinfo(FALSE)
stopCluster(cl)

from Rstudio, I can see that the size of the matrix A is ~38Mb. But when I run the script above, I find that the garbage collector reports the following values:

36.6 Mbytes of cons cells used (56%)
982.1 Mbytes of vectors used (33%)

what I don't grasp clearly is where all memory was spent. In fact, the code above runs faster with a single worker (%do%) rather than with 2 workers (%dopar%). Do you know the reason for the large memory usage of this script?

Thanks

foreach memory usage in R

0 Answers