Is there an early termination option in the parallel processing function (such as future_apply) in R?

user_9958677 0
Viewed 67

I am using the future.apply package in my R code to do parallel processing on 100 Monte-Carlo Simulations on an optimization problem.

For now let's assume that the output of the optimization for each simulation is either True (optimal solution found) or False (no optimal solution found). Can I set up condition(s) so that if the first 10 simulations return False, the parallel process will be terminated?

EDIT : I include a small example below - can I terminate this after 10 runs?

library(future.apply)

plan(multiprocess)
outputs <- future_lapply(1:100, function(i){
  temp <- FALSE
  return(temp)
})
parallel-processing r
1 Answers

You could use an intermediary file in which you write the result of each simulation. Each time a new simulation is launched, you check if the first 10 results are FALSE and if yes, you ask the thread to kill itself. future_lapply() must be used inside a try() wrapper because the function will obviously fail.

library(future.apply)
library(tools)

func <- function(i){
  tbl_results <- read.csv("results.csv")
  results <- tbl_results[, 2]
  
  if(length(results) >= 10 & all(results[1:10] == FALSE)) pskill(Sys.getpid())
  
  result <- FALSE
  
  cat(paste0(i, ",", result, "\n"), file = "results.csv", append = TRUE)
  return(result)
}


plan(multiprocess)
cat("input,result\n", file = "results.csv")
outputs <- try({future_lapply(1:100, func)}, silent = TRUE)

if(!is.list(outputs)) stop("10 first simulations returned FALSE")
user_85125680
Related