sklearn validation score meaning

user_7042186 0
Viewed 332

Whenever i train an MLP model on sklearn, I get this output here:

from sklearn.neural_network import MLPClassifier


clf_mlp = MLPClassifier(random_state=1,\
                        max_iter=200,\
                        hidden_layer_sizes=(256,256,256),\
                        early_stopping = True,\
                        verbose=True).fit(X, pdf_train["label"])


Iteration 1, loss = 1.23744239
Validation score: 0.649914
Iteration 2, loss = 1.07239263
Validation score: 0.652249
Iteration 3, loss = 0.99360697
Validation score: 0.652205
Iteration 4, loss = 0.90097632
Validation score: 0.646963
<And it goes on...>

I`m confused about how to read this log: Is the "loss" there a Training Loss or Validation loss? And is the "Validation score" value accuracy or is it validation loss?

If you could point where in sklearn documentation this is explained, I would be grateful as well.

scikit-learn python
2 Answers

It looks like the loss that is printed is the training loss (default: log-loss) as evidenced by the source code.

The Validation score that is printed is indeed the score (default: accuracy) on the validation data, as evidenced by this function in the source code.

This should definitely be better documented, feel free to submit an issue.

user_48000860

The loss is calculated on the training set (see here in source). The validation score is the accuracy on the validation set. The score function is defined from sklearn's base ClassifierMixin.

user_72266450
Related