ValueError: multiclass format is not supported for average_precision_score

Viewed 38

I am trying to calculate the average precision score for my object detection predicted labels with the ground truth labels using scikit-learn library. However, I get an error while I try to run the code.

    y_pred = np.array([106, 86, 115, 92])
    y_truth = np.array([105, 85, 114, 91])
    average_precision_score(y_pred, y_truth)

The error:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/home/lunet/conm/Desktop/Stenosis-Project/vgg16_pre_weights.ipynb Cell 34 in <cell line: 38>()
    115     y_pred = np.array([106, 86, 115, 92])
    116     y_truth = np.array([105, 85, 114, 91])
--> 117     average_precision_score(y_pred, y_truth)
    118     i = i + 1
    119 plt.show()

File ~/Mambaforge-Linux-x86_64/envs/stenosispy/lib/python3.9/site-packages/sklearn/metrics/_ranking.py:234, in average_precision_score(y_true, y_score, average, pos_label, sample_weight)
    227         raise ValueError(
    228             f"pos_label={pos_label} is not a valid label. It should be "
    229             f"one of {present_labels}"
    230         )
    231 average_precision = partial(
    232     _binary_uninterpolated_average_precision, pos_label=pos_label
    233 )
--> 234 return _average_binary_score(
    235     average_precision, y_true, y_score, average, sample_weight=sample_weight
    236 )

File ~/Mambaforge-Linux-x86_64/envs/stenosispy/lib/python3.9/site-packages/sklearn/metrics/_base.py:72, in _average_binary_score(binary_metric, y_true, y_score, average, sample_weight)
     70 y_type = type_of_target(y_true)
     71 if y_type not in ("binary", "multilabel-indicator"):
---> 72     raise ValueError("{0} format is not supported".format(y_type))
     74 if y_type == "binary":
     75     return binary_metric(y_true, y_score, sample_weight=sample_weight)

ValueError: multiclass format is not supported
1 Answers

Average precision is basically the area under precision/recall curve. It operates on scores (e.g. predict_proba() results or pre-argmax'ed NN outputs), not predicted labels. Additionally, in a multiclass case you'll likely need to run label_binarize() on y_truth.

Related