GridsearchCV is giving score as nan

user_15093574 0
Viewed 263

I am having trouble to use GridsreachCV class to output the best parameters in my project. I created a class and include my own scoring method. I tried to search for my solution, but probably my lack of knowledge could not figure out the problem. The question is,

  • why am I getting score=nan?
  • TypeError: fit() missing 1 required positional argument: 'y'

I have created a toy example to reproduce the error, please ignore for the logical output. Here is code

import numpy as np
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import make_scorer
from  sklearn.base import BaseEstimator
from sklearn.model_selection import train_test_split

class Foo(BaseEstimator):
  def __init__(self, start=0, end=0):
    self.start = start
    self.end = end

  def fit(self, X, y):
    X_, X_val, y_, y_val = train_test_split(X, y, test_size=0.20, random_state=42) 

    for i in range(self.start, self.end):
        X[i] = X[i]**2*y[i]
    return X

  def predict(self, X):
    val = np.max(X)
    return val

  def accuracy(self, x):
    if x > 50:
      return 100
    else:
      return 1

#=======================================================================
X = np.array(np.random.random(200)*100)
y = np.array(np.random.randint(2,size=200))
param_grid = {'start':[0, 10, 50], 'end':[60, 80, 100]}

foo = Foo()
scoring = make_scorer(foo.accuracy, greater_is_better=False)
grid = GridSearchCV(foo, param_grid=param_grid, scoring=scoring, verbose = 3, cv=2, refit=True) 
grid.fit(X, y) #fixed
print(grid.best_params_) 
#=======================================================================

Here is the error:

Fitting 2 folds for each of 9 candidates, totalling 18 fits
[CV 1/2] END .....................end=60, start=0;, score=nan total time=   0.0s
[CV 2/2] END .....................end=60, start=0;, score=nan total time=   0.0s
[CV 1/2] END ....................end=60, start=10;, score=nan total time=   0.0s
[CV 2/2] END ....................end=60, start=10;, score=nan total time=   0.0s
[CV 1/2] END ....................end=60, start=50;, score=nan total time=   0.0s
[CV 2/2] END ....................end=60, start=50;, score=nan total time=   0.0s
[CV 1/2] END .....................end=80, start=0;, score=nan total time=   0.0s
[CV 2/2] END .....................end=80, start=0;, score=nan total time=   0.0s
[CV 1/2] END ....................end=80, start=10;, score=nan total time=   0.0s
[CV 2/2] END ....................end=80, start=10;, score=nan total time=   0.0s
[CV 1/2] END ....................end=80, start=50;, score=nan total time=   0.0s
[CV 2/2] END ....................end=80, start=50;, score=nan total time=   0.0s
[CV 1/2] END ....................end=100, start=0;, score=nan total time=   0.0s
[CV 2/2] END ....................end=100, start=0;, score=nan total time=   0.0s
[CV 1/2] END ...................end=100, start=10;, score=nan total time=   0.0s
[CV 2/2] END ...................end=100, start=10;, score=nan total time=   0.0s
[CV 1/2] END ...................end=100, start=50;, score=nan total time=   0.0s
[CV 2/2] END ...................end=100, start=50;, score=nan total time=   0.0s

/usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_validation.py:372: FitFailedWarning: 
18 fits failed out of a total of 18.
The score on these train-test partitions for these parameters will be set to nan.
If these failures are not expected, you can try to debug them by setting error_score='raise'.

Below are more details about the failures:
--------------------------------------------------------------------------------
18 fits failed with the following error:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/dist-packages/sklearn/model_selection/_validation.py", line 678, in _fit_and_score
    estimator.fit(X_train, **fit_params)
TypeError: fit() missing 1 required positional argument: 'y'

  warnings.warn(some_fits_failed_message, FitFailedWarning)
gridsearchcv scikit-learn machine-learning python
1 Answers

You get TypeError: fit() missing 1 required positional argument: 'y' as an error, because you forgot to pass y in grid.fit(X). It could be that solving this issue resolves your score being NaN.

In fact, the output also tells you the following:

The score on these train-test partitions for these parameters will be set to nan.
If these failures are not expected, you can try to debug them by setting error_score='raise'.

Finally, I would suggest that you make sure that you don't accidentally overflow while calculating X[i] = X[i]**2*y[i].

EDIT:
In a comment you mentioned that you now get the error TypeError: accuracy() takes 2 positional arguments but 3 were given. Your accuracy method is used in make_scorer(). If you look at the documentation, you'll see that a score function with signature score_func(y, y_pred, **kwargs) is expected, i.e. you are missing an argument in your accuracy method (self is not y). Additionally, as accuracy doesn't use self in the first place, I would suggest moving it out of the class, making it a "normal" function.

Finally, I get the feeling that you are unfamiliar or not confident with Python. I'd suggest that you invest some time into really understanding what these errors mean and also how to use external packages like numpy and sklearn (by reading the documentation ;)).

user_132465740
Related