Geometric mean of precision and recall

user_13339065 0
Viewed 917

I am trying to make a custom metric to evaluate a ML model. I want to make the Weighted geometric mean of precision and recall and weigh it in a way that prioritizes recall over precision. I know that the geometric mean is the sqrt(precision * recall) but I'm not sure how to parametererize it to give more importance to recall in python. There is this metric from imbalance library but I don't see any weights that I can Provide

imblearn.metrics.geometric_mean_score(y_true, y_pred, labels=None, pos_label=1, average='multiclass', sample_weight=None, correction=0.0)

Any idea on how to implement what I want in python?

classification metrics machine-learning python-3.x python
1 Answers

If you are facing a class-imbalance issue, a multi-class geometric mean between precision and recall, weighted by label supports (number of position samples for each label) is a great option (this is allowed in imblearn API that you have linked, with parameter average='weighted').

However, IIUC, is not what you are looking for. You are trying to take a weighted geometric mean between precision and recall.

I couldn't find any implementations for weighted geometric mean in popular libraries so I wrote a custom function for this.

You can calculate precision and recall using the sklearn api from y_true and y_pred and then use the function to calculate the weighted geometric mean.

I have written the weighted_geometric_mean function based on the following definition (the first form with powers instead of exponentials) -

enter image description here

from sklearn.metrics import precision_score, recall_score

y_true = [0, 1, 2, 0, 1, 2]
y_pred = [0, 2, 1, 0, 0, 1]

precision = precision_score(y_true, y_pred, average='micro')
recall = recall_score(y_true, y_pred, average='micro')
#parameter 'micro' calculates metrics globally by counting the total TP, FN and FP

scores = [precision, recall]
weights = [0.6,0.4]  #60% precision, 40% recall

def weighted_geometric_mean(scores, weights):
    wgm = np.product(np.power(scores, weights))
    return wgm

weighted_geometric_mean(scores, weights)

0.3333333333333333

The above implementation uses global precision and recall with the parameter micro. If you want to consider class-wise weights to calculate precision and recall (for class imbalance situations), please set it to weighted

Edit: On a side note, weighted geometric average between global precision and recall with weights that sum up to 1 (60:40 or 50:50 etc) will always result in the same final value! You can derive this by writing precision in its TP, FP form, and same for Recall. I would therefore recommend label support weighted precision and recall.

user_47559540
Related