9
$\begingroup$

Consider the example in this article http://text-analytics101.rxnlp.com/2014/10/computing-precision-and-recall-for.html Will accuracy be (30 + 60 + 80)/300? what is weighted precision?

$\endgroup$
2
  • 1
    $\begingroup$ en.wikipedia.org/wiki/Confusion_matrix gives you formula for accuracy. You'll just need to substitute the values. $\endgroup$
    – SmallChess
    Oct 7 '17 at 10:57
  • $\begingroup$ @SmallChess so the accuracy is calculated separately for each class? $\endgroup$
    – farheen
    Oct 7 '17 at 11:04
6
$\begingroup$

I've got a wonderful solution and a perfect understandable solution for this problem as I was looking for same from this Question

You can calculate and store accuracy with:

(accuracy <- sum(diag(mat)) / sum(mat))
# [1] 0.9333333

Precision for each class (assuming the predictions are on the rows and the true outcomes are on the columns) can be computed with:

(precision <- diag(mat) / rowSums(mat))
#     setosa versicolor  virginica 
#  1.0000000  0.9090909  0.8750000 

If you wanted to grab the precision for a particular class, you could do:

(precision.versicolor <- precision["versicolor"])
# versicolor 
#  0.9090909 

Recall for each class (again assuming the predictions are on the rows and the true outcomes are on the columns) can be calculated with:

 recall <- (diag(mat) / colSums(mat))
    #     setosa versicolor  virginica 
    #  1.0000000  0.8695652  0.9130435 

If you wanted recall for a particular class, you could do something like:

(recall.virginica <- recall["virginica"])
# virginica 
# 0.9130435 

If instead you had the true outcomes as the rows and the predicted outcomes as the columns, then you would flip the precision and recall definitions.

Data:

(mat = as.matrix(read.table(text="  setosa versicolor virginica
 setosa         29          0         0
 versicolor      0         20         2
 virginica       0          3        21", header=T)))
#            setosa versicolor virginica
# setosa         29          0         0
# versicolor      0         20         2
# virginica       0          3        21
$\endgroup$
2
$\begingroup$

Accuracy is for the whole model and your formula is correct.

Precision for one class 'A' is TP_A / (TP_A + FP_A) as in the mentioned article. Now you can calculate average precision of a model. There are a few ways of averaging (micro, macro, weighted), well explained here:

'weighted': Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; (...)

$\endgroup$
0
$\begingroup$

I think your confusion come from the 3x3 table. But ... the link has an example on precision and recall for Label A. Accuracy is very similar.

Accuracy for A = (30 + 60 + 10 + 20 + 80) / (30 + 20 + 10 + 50 + 60 + 10 + 20 + 20 + 80)

https://en.wikipedia.org/wiki/Confusion_matrix

I don't know what weighted precision is about.

$\endgroup$
4
  • $\begingroup$ I do understand the denominator which is N and in numerator 30 + 60 + 80 are examples that were classified correctly, can you explain 10 + 20 in numerator? $\endgroup$
    – farheen
    Oct 7 '17 at 11:21
  • $\begingroup$ @farheen I merely followed the formula. 60+10+20+80 = TN for label A. $\endgroup$
    – SmallChess
    Oct 7 '17 at 11:23
  • $\begingroup$ They are cases that predicted for B and C, but the true labels are not A. (TN for A) $\endgroup$
    – SmallChess
    Oct 7 '17 at 11:23
  • $\begingroup$ So we calculate accuracy for each label separately? then what will be the accuracy for entire model? $\endgroup$
    – farheen
    Oct 7 '17 at 11:26
0
$\begingroup$

Try PyCM, it gives you accuracy and other parameters.

PyCM is a multi-class confusion matrix library written in Python

... and a proper tool for post-classification model evaluation that supports most classes and overall statistics parameters.

Check the html version of output.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.