How to transform '2 levels ParamUty' class in nested cross-validation of mlr3proba?

Question

For survival analysis, I am using mlr3proba package of R.
My dataset consists of 39 features(both continuous and factor, which i converted all to integer and numeric) and target (time & status).
I want to tune hyperparameter: num_nodes, in Param_set.
This is a ParamUty class parameter with default value: 32,32.
so I decided to transform it.
I wrote the code as follows for hyperparamters optimization of surv.deephit learner using 'nested cross-validation' (with 10 inner and 3 outer folds).

#task definition
task.mlr <- TaskSurv$new(id = "id", backend = main.dataset, event = 'status', time = 'time')

#learner definition
dh.learner <- lrn('surv.deephit') 
   
#resampling method
resampling <- rsmp('cv', folds =10)
  
#tuner method
tuner <- tnr('random_search')

#measure method
measure <- msr('surv.harrellC')

#termination method
terminator <- trm('stagnation')

#search_space definition(for num_nodes)
search_space <- ps(num_nodes = p_fct(list(c(32,64,128,256)), trafo = function(x) c(sample(x,1), sample(x,1))))

#To check search_space
generate_design_random(search_space,10)$transpose()

When I ran the last lines of code with transpose, it demonstrated a list of num_nodes each contained a paired categories, as follows:

[[1]]$num_nodes
[1] 64 128

[[2]]$num_nodes
[1] 32 256

...

then I wrote the following code:

#defining autotuner
at <- AutoTuner$new(dh.learner, resampling, measure, terminator, tuner, search_space)

#outer cross validation
resampling_outer <- rsmp('cv', folds = 3)

# nested resampling
nest_rsm <- resample(task.mlr, at, resampling_outer)

But in the result of nested resampling, instead of pair of nodes, it shows num_nodes as c(32,64,128,256). Something like this:

    num_nodes 
   c(32,64,128,256)
    num_nodes
   c(32,64,128,256)
   ...

How can I transform 'Param_Uty' with 2 levels(eg. 32,64)?
Honestly I have heavily searched this topic, and after all I did not find appropriate answer, so i appreciate your kind help.

user_11815170 · Answer

Hi thanks for using mlr3proba. I have actually just finished writing a tutorial that answers exactly this question! It covers training, tuning, and evaluating the neural networks in mlr3proba. For your specific question, the relevant part of the tutorial is this:

library(paradox)
search_space = ps(
  nodes = p_int(lower = 1, upper = 32),
  k = p_int(lower = 1, upper = 4)
)

search_space$trafo = function(x, param_set) {
  x$num_nodes = rep(x$nodes, x$k)
  x$nodes = x$k = NULL
  return(x)
}

Here I am tuning over two new hyper-parameters, one which represents number of nodes per layer and one which is the number of layers, I then create a transformation that combines them into the desired hyper-parameter, num_nodes.

You should be able to apply this to your example.

How to transform '2 levels ParamUty' class in nested cross-validation of mlr3proba?

1 Answers