How to get confusion matrix in LogisticRegressionCV?

user_3704712 0
Viewed 248

I implemented two versions of the logistic regression model. In the second version my goal is to find the best hyperparameter C, otherwise I am okay with the first version. I also need mean and std. dev. of confusion matrix and coefficient matrix for the best hyperparameter C. But I am confused how to get those in LogisticRegressionCV second version.

Version 1

    sss = RepeatedStratifiedKFold(n_splits=K_fold, n_repeats=n_repeats ,random_state=36851234)
    lambda_c=1.0
    cmn=[]
    coef=[]
    for train_index, test_index in sss.split(X,y):
        x_train,x_test=X[train_index],X[test_index]
        y_train,y_test=y[train_index],y[test_index]
        log_reg_model = LogisticRegression(max_iter=50000,C=lambda_c,penalty='l1',multi_class='ovr',class_weight='balanced',solver='liblinear')
        pipe=Pipeline([  ('polynomial_features',polynomial),   ('StandardScaler',StandardScaler()), ('logistic_regression',log_reg_model)])
        pipe.fit(x_train, y_train)
        y_pred=pipe.predict(x_test)
        y_prob = pipe.predict_proba(x_test)

        LR= pipe.named_steps['logistic_regression']
        coef.append(LR.coef_)
        cmn.append(confusion_matrix(y_test,y_pred,normalize='true'))

    cmn_std=np.std(np.array(cmn),axis=0)
    coef_std=np.std(np.array(coef),axis=0)
    cmn=np.mean(np.array(cmn),axis=0)
    coef=np.mean(np.array(coef),axis=0)

Version 2

    sss = RepeatedStratifiedKFold(n_splits=K_fold, n_repeats=n_repeats ,random_state=36851234)
    lambda_c=list(np.power(10.0, np.arange(-10, 3)))
    scoring='precision_weighted'
    log_reg_model = LogisticRegressionCV(max_iter=50000,fit_intercept=False,cv=sss,Cs=lambda_c,penalty='l1',multi_class='ovr',scoring=scoring,class_weight='balanced',solver='liblinear')
    pipe=Pipeline([  ('polynomial_features',polynomial),   ('StandardScaler',StandardScaler()), ('logistic_regression',log_reg_model)])
    pipe.fit(X,y)
    poly = pipe.named_steps['polynomial_features']
    LR= pipe.named_steps['logistic_regression']
    LR.coef_ # the shape is [3,6]; #class = 3 and #features = 6 
    LR.coefs_paths_  # the shape is [500,13,6]; #cv = 500, #C=13, #features=6

How to get mean and std. dev. of confusion matrix and coeff. matrix in second version of the model? This is not clear to me. I also see LR.C_=[100. 0.1   0.1] in the output for my data which has 3 classes. Why do we have different value of hyperparameter for each class. This part also I don't understand fully. Thanks.

logistic-regression scikit-learn python
2 Answers

This works if being the goal,

from sklearn.datasets import make_classification
import numpy as np
from sklearn.model_selection import RepeatedStratifiedKFold
from sklearn.model_selection import cross_val_predict
from sklearn.linear_model import LogisticRegressionCV
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.preprocessing import PolynomialFeatures
from sklearn.metrics import confusion_matrix

cv = RepeatedStratifiedKFold(n_splits=5, n_repeats=10 ,random_state=36851234)

x, y = make_classification(100, 6, n_classes=3, n_informative = 3)
lambda_c=list(np.power(10.0, np.arange(-10, 3)))

scoring='precision_weighted'
log_reg_model = LogisticRegressionCV( max_iter=5000,
                                      fit_intercept=False,
                                      cv=cv,
                                      Cs=lambda_c,penalty='l1',
                                      multi_class='ovr',
                                      #scoring=scoring,
                                      class_weight='balanced',
                                      solver='liblinear')

polynomial = PolynomialFeatures(2)
pipe=Pipeline([  ('polynomial_features',polynomial),   ('StandardScaler',StandardScaler()), ('logistic_regression',log_reg_model)])

pipe.fit(x, y)

LR= pipe.named_steps['logistic_regression']

y_pred = cross_val_predict(pipe, x, y, cv=10)
conf_mat = confusion_matrix(y, y_pred)


print(conf_mat)
array([[17,  8,  9],
       [ 5, 28,  0],
       [ 3,  3, 27]])
user_56122590

You have three values in LR.C_ because you are using the option multi_class='ovr' in the logistic regression. According to the scikit-learn documentation, it does one versus the rest, i.e. you have 3 classifiers in fact. See the doc sklearn.linear_model.LogisticRegression:

In the multiclass case, the training algorithm uses the one-vs-rest (OvR) scheme if the ‘multi_class’ option is set to ‘ovr’

user_42920330
Related