I am having a hard time figuring out the algorithm sklearn uses to calculate the predicted probabilities when the LogisticRegression instance is created with multi_class="multinomial". This is how I setup the logistic regression:
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
from sklearn.linear_model import LogisticRegression
clf1 = LogisticRegression(multi_class='ovr')
clf2 = LogisticRegression(multi_class='multinomial')
clf1.fit(X, y)
clf2.fit(X, y)
pred_prob_ovr = clf1.predict_proba(X)
pred_prob_multinomial = clf2.predict_proba(X)
I can reconstruct the probabilities when multi_class="ovr" using:
import numpy as np
logit = np.matmul(X, clf1.coef_.transpose()) + clf1.intercept_
A = np.exp(logit)
P = 1/(1+1/A)
Prob_ovr = P/P.sum(axis=1).reshape((-1, 1))
However, when I use multi_class="multinomial", the above procedure doesn't work and I need to use this method:
logit = np.matmul(X, clf2.coef_.transpose()) + clf2.intercept_
from sklearn.utils.extmath import softmax
Prob_multinomial = softmax(logit)
Could you please describe why the coefficients and intercept, obtained from the model with multi_class="multinomial", don't work with the first procedure?
Thanks