What do the cv.values of sklearn's RidgeCV mean when scoring = 'r2_score'?

user_12349675 0
Viewed 551

I am trying to make sense of the values stored in Scikit-Learn's RidgeCV's cv.values_ object when scoring is set to the r2_score metric.

Per the documentation for Scikit-Learn's RidgeCV function, when store_cv_values=True:

Cross-validation values for each alpha (only available if store_cv_values=True and cv=None). After fit() has been called, this attribute will contain the mean squared errors (by default) or the values of the {loss,score}_func function (if provided in the constructor).

Though I'm somewhat unclear as to the specifics of how RidgeCV's native generalized cross-validation works, if it is indeed an approximation of leave-one-out-cross validation, then what the cv_values object seems to represent is the 'r2_score' for individual (left-out) samples... Except that r2_score does not work for individual samples. What then is returned in the cv_values object when scoring is set to r2_score?

scikit-learn python
2 Answers

In short, this Generalized Cross-Validation makes the leave-one-out predictions on the entire training set, then applies the scoring function to those (rather than scoring first then averaging).

You can see that in the code, though it's a little obfuscated by the IdentityRegressor/Classifier. That line is really just finding the score with inputs predictions_ and y_. A few lines up you can see where they generate the predictions: that's where they use the trick that makes Generalized Cross-Validation an efficient way to do Leave-One-Out in the context of Ridge Regression. If you're interested, the docs link to some a report and course slides describing why/how that actually works.

user_104958930

Update: After some helpful back and forth, it seems the mystery here is partially resolved and points to an error in the sklearn documentation for the RidgeCV function. If a scoring argument is provided, the cv_values_ object returns predictions per point. (If a scoring argument is not provided, the object returns as stated with the squared error per point.)

From the source code:

Scikit-Learn RidgeCV cv_values_ object definition

user_123496750
Related