Importing KerasRegresor from Sci-Keras rather than Keras changes the results of my cross_val_score. How do I interpret the new numbers?

user_15361720 0
Viewed 4045

I'm running a cross validation to test my neural network. Before installing TensorFlow 2.0, I used

from keras.wrappers.scikit_learn import KerasRegressor
...
estimators.append(('mlp', KerasRegressor(build_fn=baseline_model, epochs=20, batch_size=50, verbose=0)))

and the results from the cross val test are reasonable Mean Squared Error numbers:

[  -31.93300056   -59.4023819    -21.60458565    -8.83761978
 -123.875         -3.63722825   -17.61767197   -30.63367081
    -235.83544922   -98.66159058]

however, after installing TensorFlow2.0, I got the warning: " DeprecationWarning: KerasRegressor is deprecated, use Sci-Keras (https://github.com/adriangb/scikeras) instead."

so I installed scikeras, and use

import scikeras
from scikeras.wrappers import KerasRegressor
...
estimators.append(('mlp', KerasRegressor(model=baseline_model, epochs=20, batch_size=50, verbose=0)))

but the results from the cross val test look like this:

[-0.00337219 -0.02298305 -0.03263641  0.03091023  0.04092502  0.02182378
 0.01505978  0.00197443  0.28461275  0.01479353]

which are unrealistic Mean Squared Error numbers.

What is happening and why these numbers change so much?

keras tensorflow machine-learning python
1 Answers

It seems that the scores you get are normalized. I have not checked the Keras Wrapper to see how it works but in the wrapper of scikeras it is mentioned that :

KerasRegressor uses sklearn_r2_score by default.

Maybe that explains why all the scores are between 0-1.

user_122694750
Related