finding class-wise accuracy in pandas using value_counts()

user_1576804 0
Viewed 1023

I have two columns in a pandas dataframe : label(ground truth) and pred (model prediction). I have 4 classes in the labels: dog, cat, elephant and snake. What I want is the accuracy or precision of the prediction per class. For instance if I have dataframe below:

label    pred
dog      cat
elephant elephant
dog    snake
cat     cat
snake   snake
snake   cat
dog    dog

What I do is use value_counts for each class and then manually plug in the ratios to get the accuracy in pandas. The problem is that value_counts is sorted by raw count numbers so the order for label and pred can be different.

numerators = df[pred].value_counts()
   denominators = df[label].value_counts()

and then I get the outputs:

 dog    0.33
  cat   1
  snake  0.5
  elephant 1

Is there a way to automate this in pandas?

pandas python
4 Answers

Ben's answer solves your problem promptly. I would just want to add the confusion matrix:

confusion_matrix = (df.groupby('label')['pred']
                      .value_counts(normalize=True)
                      .unstack(fill_value=0)
                   )

Output:

pred           cat       dog  elephant     snake
label                                           
cat       1.000000  0.000000       0.0  0.000000
dog       0.333333  0.333333       0.0  0.333333
elephant  0.000000  0.000000       1.0  0.000000
snake     0.500000  0.000000       0.0  0.500000
user_42384080

you can check where both columns are equals and then groupby the first column and mean:

print (df['label'].eq(df['pred']).groupby(df['label']).mean())
label
cat         1.000000
dog         0.333333
elephant    1.000000
snake       0.500000
dtype: float64
user_92747320

Let us try pd.crosstab, after you get the matrix , we can do heat map to see the correlation 

pd.crosstab(df.label,df.pred,normalize='index')
pred           cat       dog  elephant     snake
label                                           
cat       1.000000  0.000000       0.0  0.000000
dog       0.333333  0.333333       0.0  0.333333
elephant  0.000000  0.000000       1.0  0.000000
snake     0.500000  0.000000       0.0  0.500000
user_79645270

Since you asked for accuracy or precision I suggest going with sklearn.metrics.classification_report:

from sklearn.metrics import classification_report

print(classification_report(df['label'].values, df['pred'].values))

enter image description here

user_50436930
Related