How to set specific values in `paradox`?

user_1297830 0
Viewed 225

Is there a way to set particular values of parameters in the R package paradox? Say I do hyperparameter tuning for a random forest method and I want to test for mtry = c(2, 3, 7, 8) and min.node.size = c(2, 5, 7), i.e., a 4 x 3 grid with non-equal distances between the values.

Currently, I have to do a large 7 x 6 grid search to include these values, testing combinations that I'm not interested in:

tuner_params = ParamSet$new(list(
  ParamInt$new("mtry", lower = 2, upper = 7),
  ParamInt$new("min.node.size", lower = 2, upper = 6)
))

generate_design_grid(tuner_params, param_resolutions = c(mtry = 7, min.node.size = 5))
grid-search hyperparameters mlr3 gridsearchcv
1 Answers

One way to overcome this is to not use grid search but TunerDesignPoints.

See example:

library(paradox)
library(mlr3)
library(mlr3tuning)
library(mlr3learners)
library(data.table)

tuner_params = ParamSet$new(list(
  ParamInt$new("mtry", lower = 2, upper = 8),
  ParamInt$new("min.node.size", lower = 2, upper = 7)
))

Specify custom design points:

design = data.table(expand.grid(mtry = c(2, 3, 7, 8),
                                min.node.size = c(2, 5, 7)))

tuner = tnr("design_points", design = design)

sonar_task = tsk("sonar")
r_lrn  = lrn("classif.ranger", predict_type = "prob")

instance = TuningInstance$new(
  task = sonar_task,
  learner =  r_lrn,
  resampling = rsmp("cv", folds = 3),
  measures = msr("classif.acc"),
  param_set = tuner_params,
  terminator = term("none")) #no terminator since you want all design points evaluated


tuner$tune(instance)

instance$archive()

#output

    nr batch_nr  resample_result task_id     learner_id resampling_id iters params tune_x warnings errors classif.acc
 1:  1        1 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8462388
 2:  2        2 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8366460
 3:  3        3 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8317460
 4:  4        4 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8269151
 5:  5        5 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8366460
 6:  6        6 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8173913
 7:  7        7 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8221532
 8:  8        8 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8124914
 9:  9        9 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8415459
10: 10       10 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8173223
11: 11       11 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8221532
12: 12       12 <ResampleResult>   sonar classif.ranger            cv     3 <list> <list>        0      0   0.8221532

12 points evaluated like we specified in the design grid.

user_85730680
Related