Calculating Equal error rate(EER) for a multi class classification problem

Your question is related to this one: ROC for multiclass classification , since EER (equal error rate) is calculated from the ROC by adjusting acceptance threshold.

I will review the conceptual process of what is EER for multiclass classification. Suppose you have n > 2 classes, for example: A, B, C and set of samples x in X with their true labels. The idea is to binarize the problem by converting it to n binary classification problems: for each class (say A) and sample (say x) there are 2 possibilities: x is in A, or x is not in A. If a in A is classified in class A, this is a true positive, if a is classified in B or C, it is false rejection or false negative. Similarly, if b in B is classified in A, it is a false acceptance or false positive. Then for each class you can compute FAR (false acceptance rate) and FRR (false rejection rate), adjust thresholds and compute EER for each class. Then you can average over the EERs of each classes. Another approach is to calculate FRR and FAR, first average them and then adjust the parameters so average FRR and average FAR are the same (this is more complicated).

An introduction for the "One vs. Rest" strategy can be found in TowardsDataScience and Data Science Stack Exchange.

Open issues:

1. In binary classification the meaning of threshold is clear. In multiclass classification it is not clear. sklearn probably handles it in the background when plotting the ROC.
2. How to implement both binary EER and non-binary EER in TensorFlow, for efficient training of deep neural networks.

I hope it helps, and I will be happy to see comments and additions which will make this issue clearer and answer the open issues I wrote above.