3
$\begingroup$

I'm trying to find the True negative in a confusion matrix, I have computed successfully from scratch the precision and recall/sensibility, now i need to compute the accuracy and specificity. This is my confusion Matrix:

cm

Computing the precision for 0 class

Precision=  TP*100/(TP+FP)
precision = 66*100/(66+2+0)
precision = 97.0588

Computing the precision for 1 class

Precision=  TP*100/(TP+FP)
precision = 81*100/(81+1+1)
precision = 97.5903

Computing the precision for 2 class

Precision=  TP*100/(TP+FP)
precision = 56*100/(56+3+0)
precision = 94.9152

Using the Pycm library I got:

PPV(Precision or positive predictive value) 0.97059  0.9759 0.94915

where 0.97059 is the precision for the class 0, and the next for 1 and the last for the 2 class.

Computing the recall for 0 class

recall =  TP*100/(TP+FN)
recall =  66/(66+2+0)
recall =  97.0588

Computing the recall for 1 class

recall =  TP*100/(TP+FN)
recall =  81/(81+2+3)
recall =  94.1860

Computing the recall for 2 class

recall =  TP*100/(TP+FN)
recall =  56/(56+0+1)
recall =  94.1860

Using Pycm library I got:

TPR(recall or true positive rate)  0.98507 0.94186  0.98246

where 0.98507 is the precision for the class 0, and the next for 1 and the last for the 2 class.

What happen now if I want to compute the accuracy? equation: Accuracy = (TP+TN)*100/(TP+TN+FP+FN) the equation is ok? I'm using the constant 100 to get the percent and not 0.x or 0.00xx but 90.x etc.

accuracy equation

I would like to know how I can get the True Negatives (TN) to compute the accuracy and specificity, currently using Pycm library I'm getting this values for the 3 classes:

ACC(Accuracy) 0.98571       0.96667       0.98095
$\endgroup$
3
$\begingroup$

I'm one of the PyCM developers.

  1. Your precision calculation method is completely correct.
  2. For recall calculation you should consider improperly classified items in each row :

class 0 :

recall =  TP*100/(TP+FN)
recall =  66/(66+1+0)
recall =  98.5074

class 1 :

recall =  TP*100/(TP+FN)
recall =  81/(81+2+3)
recall =  94.1860

class 2 :

recall =  TP*100/(TP+FN)
recall =  56/(56+0+1)
recall =  98.2456
  1. Accuracy formula is correct.
  2. Finding TN for each class :

You should consider class vs other and add up items that classified correctly as other, in other words eliminate row and col related to class and add up remaining.

class 0 :

accuracy = (TP+TN)*100/(TP+TN+FN+FP)
accuracy = (66+141)/(66+141+2+1)
accuracy = 98.5714

class 1 :

accuracy = (TP+TN)*100/(TP+TN+FN+FP)
accuracy = (81+122)/(81+122+2+5)
accuracy = 96.6666

class 2 :

accuracy = (TP+TN)*100/(TP+TN+FN+FP)
accuracy = (56+150)/(56+150+1+3)
accuracy = 98.0952

Best Regards

Sepand Haghighi

$\endgroup$
  • 1
    $\begingroup$ Hi, Sepand thanks so much for your reply, really you made a great work creating PyCM library, it will help to us to minimize the time to evaluate machine learning models, however in my case I'm doing a thesis and "I need to do manual calculations" to demostrate to the jury.. anyways.. I would like to know why you don't you sum all the TP diag and then subtract it with the TP of the actual class, I was doing that lately.. I find those trick here: bit.ly/2wyBtJ3 the Prof. Ahmad Hassanat said that. anyways I think that your method is more precise and is according with the PyCM library. $\endgroup$ – Freddy Daniel Jun 3 at 0:06
  • $\begingroup$ I read the link you mentioned above. Prof. Ahmad Hassanat method about TN calculation is not correct. Because when you have a multi-class classification problem and want to calculate TN for each class you should consider confusion matrix in binary form (class and other) and any item that classified as other correctly (may incorrect as other classes) should be included in TN. If you need more detail I'm available in Telegram by sepkjaer id. $\endgroup$ – sepandhaghighi Jun 3 at 10:11
  • $\begingroup$ Again I found one error in this paper: researchgate.net/publication/… Pag. 6 (2.9.2 (Multi-classification example) the author of that said that the FN are compute adding the col of the actual class, which is incorrect. The FP on there yes we use the col of the class but not for FN. $\endgroup$ – Freddy Daniel Jun 4 at 23:00
  • $\begingroup$ This paper method is true, you should consider transpose of matrix. $\endgroup$ – sepandhaghighi Jun 5 at 11:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.