How to compute accuracy for multi class classification problem and how is accuracy equal to weighted precision?

Question

Consider the example in this article http://text-analytics101.rxnlp.com/2014/10/computing-precision-and-recall-for.html Will accuracy be (30 + 60 + 80)/300? what is weighted precision?

Answer 1

I've got a wonderful solution and a perfect understandable solution for this problem as I was looking for same from this Question

You can calculate and store accuracy with:

(accuracy <- sum(diag(mat)) / sum(mat))
# [1] 0.9333333

Precision for each class (assuming the predictions are on the rows and the true outcomes are on the columns) can be computed with:

(precision <- diag(mat) / rowSums(mat))
#     setosa versicolor  virginica 
#  1.0000000  0.9090909  0.8750000 

If you wanted to grab the precision for a particular class, you could do:

(precision.versicolor <- precision["versicolor"])
# versicolor 
#  0.9090909 

Recall for each class (again assuming the predictions are on the rows and the true outcomes are on the columns) can be calculated with:

 recall <- (diag(mat) / colSums(mat))
    #     setosa versicolor  virginica 
    #  1.0000000  0.8695652  0.9130435 

If you wanted recall for a particular class, you could do something like:

(recall.virginica <- recall["virginica"])
# virginica 
# 0.9130435 

If instead you had the true outcomes as the rows and the predicted outcomes as the columns, then you would flip the precision and recall definitions.

Data:

(mat = as.matrix(read.table(text="  setosa versicolor virginica
 setosa         29          0         0
 versicolor      0         20         2
 virginica       0          3        21", header=T)))
#            setosa versicolor virginica
# setosa         29          0         0
# versicolor      0         20         2
# virginica       0          3        21

Answer 2

Accuracy is for the whole model and your formula is correct.

Precision for one class 'A' is TP_A / (TP_A + FP_A) as in the mentioned article. Now you can calculate average precision of a model. There are a few ways of averaging (micro, macro, weighted), well explained here:

'weighted': Calculate metrics for each label, and find their average, weighted by support (the number of true instances for each label). This alters ‘macro’ to account for label imbalance; (...)

Answer 3

I think your confusion come from the 3x3 table. But ... the link has an example on precision and recall for Label A. Accuracy is very similar.

Accuracy for A = (30 + 60 + 10 + 20 + 80) / (30 + 20 + 10 + 50 + 60 + 10 + 20 + 20 + 80)

https://en.wikipedia.org/wiki/Confusion_matrix

I don't know what weighted precision is about.

Answer 4

Try PyCM, it gives you accuracy and other parameters.

PyCM is a multi-class confusion matrix library written in Python

... and a proper tool for post-classification model evaluation that supports most classes and overall statistics parameters.

Check the html version of output.