Dynamic library dependencies not recognized when run in parallel under R foreach()

The following works with no problems. Unlike the code in the question, it starts by detecting the number of available cores, create a cluster and make it available to foreach.

library(Rfast)
library(doParallel)
library(foreach)

cores <- detectCores()
cl <- makeCluster(cores)
registerDoParallel(cl)

set.seed(2020)
A <- matrix(rnorm(1e6), 1000, 1000)
cm <- foreach(n = 1:4, 
              .combine = rbind, 
              .packages = "Rfast") %dopar% {
  colmeans(A)
}

stopCluster(cl)

str(cm)
#num [1:4, 1:1000] -0.02668 -0.02668 -0.02668 -0.02668 0.00172 ...
# - attr(*, "dimnames")=List of 2
#  ..$ : chr [1:4] "result.1" "result.2" "result.3" "result.4"
#  ..$ : NULL

The foreach package was great for its time. But, now, parallel computations should be done with future just for the static code analysis handling the correct export to workers. As a result, under the future approach, registering a package with .packages= isn't needed. Moreover, the future mirrors usual R code with only a slight change regarding the setup of an output variable as a listenv. For example, we have:

library("future")
library("listenv")
library("Rfast")

plan(tweak(multiprocess , workers = 2L))
# For all cores, directly use:
# plan(multiprocess)

# Generate matrix once
A <- matrix(rnorm(1e6), 1000, 1000)
# Setup output
x <- listenv()

# Iterate 4 times
for(i in 1:4) {
  # On each core, compute the colmeans()
  x[[i]] %<-% {
    colmeans(A)
    # For better control over function applies, use a namespace call
    # e.g. Rfast::colmeans(A)
  }
}

# Switch from listenv to list
output <- as.list(x)

Thanks to the answers by @RuiBarradas and @coatless, I realize that the problem is not with foreach(), because (1) the problem occurred when I ran the code with future too, and (2) it occurred with the foreach() code even with the wrong call, when I didn't register the cluster. When there is no cluster registered, foreach() will throw a warning and runs in sequential mode instead. But that didn't happen.

Therefore, I realize that the problem must have occurred even before the foreach() call. In the log, it appeared right after the message Loading package RcppZiggurat. Something must have gone wrong when this package is loaded.

I then checked the dependencies of RcppZiggurat, and found that it depends on another package called RcppGSL, which interfaces R and the GSL library. Bingo, that's where libgsl.so.0 is needed when RcppZiggurat is called.

So I made an R script named test-gsl.R, which has the following two lines.

library(RcppZiggurat)
print(‘OK’)

Now, I run the following on the head node

$ module load R/3.6.3
$ Rscript test-gsl.R

And everything works fine. The ‘OK’ is printed.

But this doesn’t work if I submit the job on the compute node. First, the PBS script, called test.sh, is as follows

### Resources request
#PBS -l select=1:ncpus=1:mem=1GB

### Walltime
#PBS -l walltime=00:01:00

echo Working directory is $PBS_O_WORKDIR
cd $PBS_O_WORKDIR

### Run R
module load R/3.6.3
Rscript test-gsl.R

Then I ran

qsub test.sh

And the error popped out. This means that there is something different between the compute node and the head node on my system, and nothing to do with the packages. I contacted the system administrator, who explained to me that the GSL library is available on the head node at the default path, but not on the compute node. So in my shell script, I need to add module load gsl/2.1 before running my R script. I tested that and everything worked.

The solution seems simple enough, but I know very little about Linux administration to realize it. Only after asking around and trying (rather blindly) many things did I finally come to this solution. So thanks to those who've offered help, and mea culpa for not being able to describe the problem accurately at the beginning.