Weighted Classification Metric for Multi class classification

Kartheek Palepu 0
Viewed 35

I have a multi-class classification problem with the classes X-Small, Small, Medium and Big.

I have the following requirement:

enter image description here

Summary:

  • Predicting a X-Small as X-Small is GOOD
  • Predicting a X-Small as Small is OK
  • Predicting a X-Small as Medium is BAD
  • Predicting a X-Small as Big is BAD

Similarly

  • Predicting a Small as X-Small is OK
  • Predicting a Small as Small is GOOD
  • Predicting a Small as Medium is OK
  • Predicting a Small as Big is BAD

Similarly

  • Predicting a Medium as X-Small is BAD
  • Predicting a Medium as Small is OK
  • Predicting a Medium as Medium is GOOD
  • Predicting a Medium as Big is OK

Similarly

  • Predicting a Big as X-Small is BAD
  • Predicting a Big as Small is BAD
  • Predicting a Big as Medium is OK
  • Predicting a Big as Big is GOOD

Question

  • What is the ideal classification metric to use?
  • What is the ideal loss function to use?
classification multiclass-classification metrics machine-learning python
1 Answers

There are different alternatives, which are basically based upon binary 2x2 contingency tables: For multiclass classification, the easy way to go is to use:

  • accuracy = sum(diag(matrix))/sum(matrix). The closest to one the better.
  • But you can also use it for each class:
    • Re-label to make a 2x2 contingency table, like the class of interest against the rest, and calculate every available index, sensibility, specificity, accuracy, etc. To create the intra-class metrics.
    • Then create the global metric by the mean of the metric by the number of classes. This is usually called the macro-sensibility, macro-xxxx measurement.
Cristobal Fresno0
Related