operands could not be broadcast together with shapes (15,3) (15,) GradientBoostingClassifier

Viewed 49

I am trying to set the loss_ function given by the GradientBoostingClassifier from sklearn but it returns an error which has no sense to me:

Error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-228-587a565619f1> in <module>
      2 for i, pred in enumerate(clf.staged_predict(testX)):
      3     print(testY, pred)
----> 4     test_score[i] = clf.loss_(testY, pred)

~/anaconda3/lib/python3.8/site-packages/sklearn/ensemble/_gb_losses.py in __call__(self, y, raw_predictions, sample_weight)
    712 
    713         if sample_weight is None:
--> 714             return np.sum(-1 * (Y * raw_predictions).sum(axis=1) +
    715                           logsumexp(raw_predictions, axis=1))
    716         else:

ValueError: operands could not be broadcast together with shapes (15,3) (15,) 

It is strange because clearly the variables testY and pred, both have shapes (15,). Could someone explain this to me please?

Code to replicate the error:

from sklearn.ensemble import GradientBoostingClassifier
from sklearn.model_selection import train_test_split
from sklearn.datasets import load_iris
import numpy as np

dt = load_iris(as_frame=True)
X, Y = np.array(dt.data), np.array(data.target)

trainX, testX, trainY, testY = train_test_split(X, Y, test_size=0.1)

clf = GradientBoostingClassifier(n_estimators=200).fit(trainX, trainY)

test_score = np.empty(len(clf.estimators_))
for i, pred in enumerate(clf.staged_predict(testX)):
    print(testY.shape) # (15,)
    print(pred.shape) # (15,)
    test_score[i] = clf.loss_(testY, pred) # Error here

1 Answers

You should apply the staged_predict_proba instead of staged_predict. The loss (and hence the gradients) is calculated with respect to the prediction probability.

This should work for you:

for i, pred in enumerate(clf.staged_predict_proba(testX)):
    test_score[i] = clf.loss_(testY, pred) 
Related