How to return a single global optimum regression model in R when running an Exhaustive Regression via the regsubsets function where id = 3:15

Question

I am comparing the results of a novel procedure which is a new proposed optimal model selection technique in machine/statistical learning which is a modified version of Exhaustive Regression aka All Subsets Regression, double aka Best Subset Regression to 3 Benchmark Variable Selection methods (LASSO and both BE & FS Stepwise). So, I am trying to get my code to work on a standard ER first before modifying it.

The following code is all from the "EER script (GitHub Version)" in the Repository I have for this project:

# Load all libraries needed for this script.
# The library specifically needed to run a basic ASR is the 'leaps' library.
library(dplyr)
library(tidyverse)
library(stats)
library(leaps)
library(purrr)

directory_path <- "~/Downloads/sample obs"
filepath_list <- list.files(path = directory_path, full.names = TRUE, recursive = TRUE)

# reformat the names of each of the csv file formatted datasets
DS_names_list <- basename(filepath_list)

## Start out by running a standard ER/ASR.
ER_fits <- lapply(datasets, function(i)
  regsubsets(x = as.matrix(select(i, starts_with("X"))), 
             y = i$Y, data = i, nvmax = 15, 
             intercept = TRUE, method = "exhaustive"))

ER_fits_summary <- summary(ER_fits)
ER_fits_summary

# The All Subsets Regression we are looking for should find a
# global optimum specification between 3 and 15 predictors (by design).
ER_Coeffs <- lapply(ER_fits, function(i) coef(i, id = 3:15))

Thanks to some much appreciated answers and comments on two previous questions here, I have gotten this far, but what I need is a single optimal regression specification returned for each of the i datasets stored in the 'datasets' object. Right now, I am getting 13 optimal specifications, one for the 4 factor fitted regression (including the intercept), one for the 5 factor case, and so on up until the 16 factor fitted regression (including the intercept), so 13 different local optima for each of the i datasets in the folder "sample obs", but I need a single global optimum for each of those i datasets instead.

p.s. That is the only way I can compare the results against the optimal i regressions fitted to each of the i datasets found by the 3 benchmark methods. I think including the final line from each will help explain more than anything. For BM1 - LASSO, the last line is:

IVs_Selected_by_LASSO <- lapply(LASSO_Coeffs, function(i) names(i[i > 0]))

For MB2 - Backward Elimination Stepwise, the last line is:

Variables_selected_by_BE <- lapply(seq_along(BE_fits), 
                          \(i) names(coef(BE_fits[[i]])))

And for MB3, Forward Selection Stepwise, the last line is:

IVs_selected_by_FS <- lapply(seq_along(FS_fits), 
                                   \(i) names(coef(FS_fits[[i]])))

Because regsubsets is more similar in its output to step than it is to enet, I tried the following and got the a list of i elements where each of them is NULL:

IVs_selected_by_ASR1 <- lapply(seq_along(ASR_fits1), 
                                   \(i) names(coef(ASR_fits1[[i]], id = 3:15)))

How to return a single global optimum regression model in R when running an Exhaustive Regression via the regsubsets function where id = 3:15

0 Answers