slurmR trying to run an example job "An error has occurred when calling `silent_system2`:"

user_1731972 0
Viewed 79

I setup a slurm cluster and I can issue a srun -N4 hostname just fine.

I keep seeing "silent_system2" errors. I've installed slurmR using devtools::install_github("USCbiostats/slurmR")

I'm following the second example 3: https://github.com/USCbiostats/slurmR

here are my files

cat slurmR.R

library(doParallel)
library(slurmR)

cl <- makeSlurmCluster(4)

registerDoParallel(cl)
m <- matrix(rnorm(9), 3, 3)
foreach(i=1:nrow(m), .combine=rbind)

StopCluster(cl)
print(m)

cat rscript.slurm

#!/bin/bash
#SBATCH --output=slurmR.out

cd /mnt/nfsshare/tankpve0/
Rscript --vanilla slurmR.R

cat slurmR.out

Loading required package: foreach
Loading required package: iterators
Loading required package: parallel
slurmR default option for `tmp_path` (used to store auxiliar files) set to:
  /mnt/nfsshare/tankpve0
You can change this and checkout other slurmR options using: ?opts_slurmR, or you could just type "opts_slurmR" on the terminal.
Submitting job... jobid:18.
Slurm accounting storage is disabled
Error: An error has occurred when calling `silent_system2`:
Warning: An error was detected before returning the cluster object. If submitted, we will try to cancel the job and stop the cluster object.
Execution halted
slurm r
2 Answers

not exactly slurmR related but if the goal is slurm + cluster's... then simply use slurmR as batch processing and modify slurmR.R as such (not the same internal function, but you see how the clustering is done)

library(parallel)
primary <- 'slurmw01'
spec <- c(rep(primary, 4), rep('slurmw02', 4), rep('slurmw03', 4), rep('slurmw04',4))
cl <- makeCluster(master=primary, spec=spec)
#load libraries, load files
simpi <- function(n) {
  points <- matrix(runif(n*2), ncol=2)
  mean(rowSums(points^2) <= 1)*4
 }

clusterEvalQ(cl, {
        ## set up each worker.  Could also use clusterExport()
          library(parallel)
  })
#copy vars, functions
clusterExport(cl, ls(all.names=TRUE), envir = .GlobalEnv)
rest <- clusterApplyLB(cl, rep(1e6, 100), simpi)
print(mean(unlist(rest)))
stopCluster(cl)
user_17319720

This problem was solved in version 0.5-1. The issue was related to systems not having Slurm accounting "on." Check out the changelog here.

user_20971710
Related