Consider the example in this article http://text-analytics101.rxnlp.com/2014/10/computing-precision-and-recall-for.html Will accuracy be (30 + 60 + 80)/300? what is weighted precision?
How to compute accuracy for multi class classification problem and how is accuracy equal to weighted precision?
1$\begingroup$ en.wikipedia.org/wiki/Confusion_matrix gives you formula for accuracy. You'll just need to substitute the values. $\endgroup$– ABCDOct 7, 2017 at 10:57
$\begingroup$ @SmallChess so the accuracy is calculated separately for each class? $\endgroup$– farheenOct 7, 2017 at 11:04
I've got a wonderful solution and a perfect understandable solution for this problem as I was looking for same from this Question
You can calculate and store accuracy with:
(accuracy <- sum(diag(mat)) / sum(mat)) #  0.9333333
Precision for each class (assuming the predictions are on the rows and the true outcomes are on the columns) can be computed with:
(precision <- diag(mat) / rowSums(mat)) # setosa versicolor virginica # 1.0000000 0.9090909 0.8750000
If you wanted to grab the precision for a particular class, you could do:
(precision.versicolor <- precision["versicolor"]) # versicolor # 0.9090909
Recall for each class (again assuming the predictions are on the rows and the true outcomes are on the columns) can be calculated with:
recall <- (diag(mat) / colSums(mat)) # setosa versicolor virginica # 1.0000000 0.8695652 0.9130435
If you wanted recall for a particular class, you could do something like:
(recall.virginica <- recall["virginica"]) # virginica # 0.9130435
If instead you had the true outcomes as the rows and the predicted outcomes as the columns, then you would flip the precision and recall definitions.
(mat = as.matrix(read.table(text=" setosa versicolor virginica setosa 29 0 0 versicolor 0 20 2 virginica 0 3 21", header=T))) # setosa versicolor virginica # setosa 29 0 0 # versicolor 0 20 2 # virginica 0 3 21
Accuracy is for the whole model and your formula is correct.
Precision for one class 'A' is
TP_A / (TP_A + FP_A) as in the mentioned article. Now you can calculate average precision of a model. There are a few ways of averaging (micro, macro, weighted), well explained here:
'weighted': Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; (...)
I think your confusion come from the 3x3 table. But ... the link has an example on precision and recall for Label A. Accuracy is very similar.
Accuracy for A = (30 + 60 + 10 + 20 + 80) / (30 + 20 + 10 + 50 + 60 + 10 + 20 + 20 + 80)
I don't know what weighted precision is about.
$\begingroup$ I do understand the denominator which is N and in numerator 30 + 60 + 80 are examples that were classified correctly, can you explain 10 + 20 in numerator? $\endgroup$– farheenOct 7, 2017 at 11:21
$\begingroup$ @farheen I merely followed the formula. 60+10+20+80 = TN for label A. $\endgroup$– ABCDOct 7, 2017 at 11:23
$\begingroup$ They are cases that predicted for B and C, but the true labels are not A. (TN for A) $\endgroup$– ABCDOct 7, 2017 at 11:23
$\begingroup$ So we calculate accuracy for each label separately? then what will be the accuracy for entire model? $\endgroup$– farheenOct 7, 2017 at 11:26
Try PyCM, it gives you accuracy and other parameters.
PyCM is a multi-class confusion matrix library written in Python
... and a proper tool for post-classification model evaluation that supports most classes and overall statistics parameters.
Check the html version of output.