Nested parallelism using foreach and cv.glmnet

Viewed 31

I'm trying to apply Elastic Net to a training dataset with approx. 29000 samples and 10000 features (approx 2.4gb) to create a binary logistic regression model. Using cv.glmnet, I originally wanted to cycle through alpha values to find the best one:

fits.mye <- list()

for (i in 1:20){
  
  fit.name <- paste0("alpha", i/20)
  fits.mye[[fit.name]] <-
    cv.glmnet(mye.train.x, mye.train.y, type.measure="class", alpha=i/20, nlambda = 250,
              family="binomial")
} 

Naturally, using a for loop to do this works, but it would take a really, really long time. So, parallel processing using foreach and doParallel was suggested to me to speed things up. My initial implementation is as follows:

# Set up parallel loop backend

n.cores <- detectCores() - 4 # Total is 44 cores, using 40. Gotta be nice to others on the server
cluster <- makeCluster(n.cores, type = "PSOCK")
registerDoParallel(cl = cluster)

# Model alpha loop
myeloidfits <- foreach (i = 0:20, .packages = "glmnet" ) %dopar% {
  
  cv.glmnet(mye.train.x, mye.train.y, type.measure = "class", alpha = i/20, family = "binomial", nlambda = 250)
}

stopCluster(cl = cluster)

And even this was taking a ridiculously long time. So I got to thinking about parallel processing, and cv.glmnet's in-build parallel argument. So I'm curious, if I run the following with parallel = TRUE in cv.glmnet:

myeloidfits <- foreach (i = 0:20, .packages = "glmnet", .verbose = TRUE ) %dopar% {
  
  cv.glmnet(mye.train.x, mye.train.y, type.measure = "class", alpha = i/20, family = "binomial", nlambda = 250, 
parallel = TRUE)
}

stopCluster(cl = cluster)

With 40 clusters and only 20 iterations of the main loop (i = 0:20), since the function has to:

  1. loop through for every alpha i/20
  2. loop through 250 lambda values for each of the above
  3. do 10-fold cross-validation for each lambda

Will:

  • the remaining 20 cores be taken advantage of by cv.glmnet's parallel = TRUE argument to deal with the calculations for each iteration, as in nested parallel processing? I might not be understanding this correctly.
  • the output of the foreach loop be of the same format as that of the for loop in example 1 (i.e. a list of cv.glmnet objects, one for each alpha = i/20 value)?

Also, if anything looks wonky in the code above, please do tell me as well.

0 Answers
Related