I'm trying to apply Elastic Net to a training dataset with approx. 29000 samples and 10000 features (approx 2.4gb) to create a binary logistic regression model. Using cv.glmnet, I originally wanted to cycle through alpha values to find the best one:
fits.mye <- list()
for (i in 1:20){
fit.name <- paste0("alpha", i/20)
fits.mye[[fit.name]] <-
cv.glmnet(mye.train.x, mye.train.y, type.measure="class", alpha=i/20, nlambda = 250,
family="binomial")
}
Naturally, using a for loop to do this works, but it would take a really, really long time. So, parallel processing using foreach and doParallel was suggested to me to speed things up. My initial implementation is as follows:
# Set up parallel loop backend
n.cores <- detectCores() - 4 # Total is 44 cores, using 40. Gotta be nice to others on the server
cluster <- makeCluster(n.cores, type = "PSOCK")
registerDoParallel(cl = cluster)
# Model alpha loop
myeloidfits <- foreach (i = 0:20, .packages = "glmnet" ) %dopar% {
cv.glmnet(mye.train.x, mye.train.y, type.measure = "class", alpha = i/20, family = "binomial", nlambda = 250)
}
stopCluster(cl = cluster)
And even this was taking a ridiculously long time. So I got to thinking about parallel processing, and cv.glmnet's in-build parallel argument. So I'm curious, if I run the following with parallel = TRUE in cv.glmnet:
myeloidfits <- foreach (i = 0:20, .packages = "glmnet", .verbose = TRUE ) %dopar% {
cv.glmnet(mye.train.x, mye.train.y, type.measure = "class", alpha = i/20, family = "binomial", nlambda = 250,
parallel = TRUE)
}
stopCluster(cl = cluster)
With 40 clusters and only 20 iterations of the main loop (i = 0:20), since the function has to:
- loop through for every alpha i/20
- loop through 250 lambda values for each of the above
- do 10-fold cross-validation for each lambda
Will:
- the remaining 20 cores be taken advantage of by
cv.glmnet'sparallel = TRUEargument to deal with the calculations for each iteration, as in nested parallel processing? I might not be understanding this correctly. - the output of the
foreachloop be of the same format as that of the for loop in example 1 (i.e. a list ofcv.glmnetobjects, one for eachalpha = i/20value)?
Also, if anything looks wonky in the code above, please do tell me as well.