Is there an R package or function for tuning logistic regression hyperparameters similar to what can be done in Python? As far as I know, the glm function has no hyperparameters available to tune, even though there are several different hyperparameters for logistic regression. Is this just something that is not possible to do in R as it is in Python? Does the glm function just set them at fixed, default values and not allow the user to try different values? Is there a way to alter the original code of the glm function to specify different hyperparameters?
1 Answers
As far as I know, there are no tunable hyperparameters in glm, but there are other logistic regression functions where hyperparameters are tunable.
The tidymodels package is very convenient for machine learning using R. It streamlines hyperparameter tuning for various data preprocessing (e.g. PCA, ...) and modelling approaches (glm and many others).
You can tune the hyperparameters of a logistic regression using e.g. the glmnet method (engine), where penalty (lambda) and mixture (alpha) can be tuned.
Specify logistic regression model using tidymodels
library(tidymodels)
library(glmnet)
# show tunable hyperparameters for various logistic regression functions
show_model_info("logistic_reg")
#> Information for `logistic_reg`
#> modes: unknown, classification
#>
#> engines:
#> classification: glm, glmnet, keras, LiblineaR, spark, stan
#>
#> arguments:
#> glmnet:
#> penalty --> lambda
#> mixture --> alpha
#> LiblineaR:
#> penalty --> cost
#> mixture --> type
#> spark:
#> penalty --> reg_param
#> mixture --> elastic_net_param
#> keras:
#> penalty --> penalty
#>
#> [fit and prediction modules omitted to be more concise]
mod <- logistic_reg(mode = "classification", engine = "glmnet",
penalty = tune(), mixture = tune())
tune hyperparameters using tidymodels
## specify recipe: model formula and preprocessing steps (if any)
rec <- recipe(Group ~ Class1 + Class2, data = data)
## specify workflow (recipe and model specification)
w <- workflow(preprocessor = rec, spec = mod)
## specify resamples for cross-validation
set.seed(28483)
r <- vfold_cv(data = data, v = 10, repeats = 1, strata = "Group")
## specify tuning grid (hyperparameter search space)
g <- expand_grid(penalty = c(0,1,2),
mixture = seq(0,1,by=0.2))
## tune logistic regression model
fit_tune <- tune_grid(w, resamples = r, grid = g)
collect and plot tuning results
tune.met <- collect_metrics(fit_tune)
tune.met
#> # A tibble: 36 × 8
#> penalty mixture .metric .estimator mean n std_err .config
#> <dbl> <dbl> <chr> <chr> <dbl> <int> <dbl> <chr>
#> 1 0 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model01
#> 2 0 0 roc_auc binary 0.877 10 0.0244 Preprocessor1_Model01
#> 3 1 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model02
#> 4 1 0 roc_auc binary 0.875 10 0.0251 Preprocessor1_Model02
#> 5 2 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model03
#> 6 2 0 roc_auc binary 0.875 10 0.0251 Preprocessor1_Model03
#> 7 0 0.2 accuracy binary 0.785 10 0.0299 Preprocessor1_Model04
#> 8 0 0.2 roc_auc binary 0.876 10 0.0245 Preprocessor1_Model04
#> 9 1 0.2 accuracy binary 0.775 10 0.0271 Preprocessor1_Model05
#> 10 1 0.2 roc_auc binary 0.847 10 0.0265 Preprocessor1_Model05
#> # … with 26 more rows
ggplot(tune.met, aes(x = mixture, y = mean, colour = factor(penalty))) +
geom_line() +
facet_wrap(~.metric) +
theme_bw()
# show best
fit_tune %>% show_best(metric = "accuracy")
#> # A tibble: 5 × 8
#> penalty mixture .metric .estimator mean n std_err .config
#> <dbl> <dbl> <chr> <chr> <dbl> <int> <dbl> <chr>
#> 1 0 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model01
#> 2 1 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model02
#> 3 2 0 accuracy binary 0.785 10 0.0299 Preprocessor1_Model03
#> 4 0 0.2 accuracy binary 0.785 10 0.0299 Preprocessor1_Model04
#> 5 0 0.4 accuracy binary 0.785 10 0.0299 Preprocessor1_Model07
toy data used
library(tidyverse)
#set.seed(1)
x1 = rnorm(100, mean = 45, sd = 5); x2 = rnorm(100, mean = 50, sd = 3);
z1 = rnorm(100, mean = 13, sd = 1.3); z2 = rnorm(100, mean = 15, sd = 2.3)
Y0 = rep("No", 100);Y1 = rep("Yes",100)
a = c(x2,x1); b = c(z2,z1); c = c(Y0,Y1);
data = tibble(Group = factor(c), Class1 = a, Class2 = b)
Created on 2021-09-21 by the reprex package (v2.0.1)
Related