ValueError: Target is multiclass but average='binary'. Please choose another average setting, one of [None, 'micro', 'macro', 'weighted']

Viewed 1805

I'm working on Multiclass Classification and I'm finding accuracies of classifiers, I’m using:

model = RandomForestClassifier(random_state=2)
model.fit(X_train, y_train)
preds = model.predict(X_test)
Accuracy=accuracy_score(y_test, preds, average='micro')

Getting an error:

TypeError: accuracy_score() got an unexpected keyword argument 'average'

and when I'm using:

model = RandomForestClassifier(random_state=2)
model.fit(X_train, y_train)
preds = model.predict(X_test)
Accuracy=accuracy_score(y_test, preds)

Getting an error:

ValueError: Target is multiclass but average='binary'. Please choose another average setting, one of [None, 'micro', 'macro', 'weighted'].

Can anyone please help me with how can I find the accuracy for the Multiclass Classification problem?

Below the code I have in my function for xgboost:

scorers = {
        'f1_score':make_scorer(f1_score),
        'precision_score': make_scorer(precision_score),
        'recall_score': make_scorer(recall_score),
        'accuracy_score': make_scorer(accuracy_score)
      }
#fitting the training dataset to the model
xgb_model = XGBClassifier(n_jobs=-1, objective='multi:softmax')
#setattr(xgb_model, 'verbosity', 2)
param_dist = {'n_estimators': stats.randint(150, 1000),
              'learning_rate': stats.uniform(0.01, 0.59),
              'subsample': stats.uniform(0.3, 0.6),
              'max_depth': [3, 4, 5, 6, 7, 8, 9],
              'colsample_bytree': stats.uniform(0.5, 0.4),
              'min_child_weight': [1, 2, 3, 4]
             }

#     numFolds = 5
#     kfold_5 = cross_validation.KFold(n = len(X), shuffle = True, n_folds = numFolds)
    skf = StratifiedKFold(n_splits=3, shuffle = True)
    gridCV = RandomizedSearchCV(xgb_model, 
                             param_distributions = param_dist,
                             cv = skf,  
                             n_iter = 5,  
                             scoring = scorers, 
                             verbose = 3, 
                             n_jobs = -1,
                             return_train_score=True,
                             refit = False)
gridCV.fit(x_train,y_train)

Here is the error when trying make_scorer(f1_score,average='micro'):

TypeError                                 Traceback (most recent call last)
<ipython-input-68-8b24047fa926> in <module>
      1 print("********** Xgboost classifier *************")
      2 start_time = time.monotonic()
----> 3 y_test, xgb_predict, xgb_pred_prob = xgboost_classifier(x,y)
      4 end_time = time.monotonic()
      5 print(timedelta(seconds=end_time - start_time))

<ipython-input-67-2661dd9c3c1a> in xgboost_classifier(x, y)
     36     scorers = {
     37             'f1_score':make_scorer(f1_score,average='micro'),
---> 38             'precision_score': make_scorer(precision_score()),
     39             'recall_score': make_scorer(recall_score()),
     40             'accuracy_score': make_scorer(accuracy_score())

TypeError: precision_score() missing 2 required positional arguments: 'y_true' and 'y_pred'

I don't know why gridCV.fit(x_train, y_train) does not feed Y values to scorers?

1 Answers

I have solved the problem by adding average to F1, precision and recall. It's only accuracy which does not need this argument !

scorers = {
            'f1_score': make_scorer(f1_score, average='micro'),
            'precision_score': make_scorer(precision_score, average='micro'),
            'recall_score': make_scorer(recall_score, average='micro'),
            'accuracy_score': make_scorer(accuracy_score)
          }
Related