4
$\begingroup$

I built a confusion matrix of three class. like, (a,b,c are the class)

-   a   b   c <= predicted
a   20  5   0
b   7  18   0
c   0   0  20

Now I want to calculate precision, recall from this confusion matrix. In order to do that I need to find out true positive, true negative, false positive, and false negative.

How can I find out the values of true positive, true negative, false positive, and false negative?

$\endgroup$
  • $\begingroup$ if you found one of the answers helpful pls. accept it - Thank you. $\endgroup$ – vonjd Sep 26 '16 at 13:29
3
$\begingroup$

When I understand your question correctly you are asking which class is the positive one and which is the negative one.

The answer is that this is to a certain extent arbitrary, so you have to decide that considering the problem at hand.

From "Machine Learning with R" by Brett Lantz, 2.nd edition, 2015, p. 318:

The most common performance measures consider the model's ability to discern one class versus all others. The class of interest is known as the positive class, while all others are known as negative.

The use of the terms positive and negative is not intended to imply any value judgment (that is, good versus bad), nor does it necessarily suggest that the outcome is present or absent (such as birth defect versus none). The choice of the positive outcome can even be arbitrary, as in cases where a model is predicting categories such as sunny versus rainy or dog versus cat.

The relationship between the positive class and negative class predictions can be depicted as a 2 x 2 confusion matrix that tabulates whether predictions fall into one of the four categories:

True Positive (TP): Correctly classified as the class of interest
True Negative (TN): Correctly classified as not the class of interest
False Positive (FP): Incorrectly classified as the class of interest
False Negative (FN): Incorrectly classified as not the class of interest

This is the reason that you e.g. have to specify the positive class when using generic performance measure functions, like ConfusionMatrix in the caret package in R.

Now another complicating factor is of course your multiclass setting, but this is answered here:
How to compute precision/recall for multiclass-multilabel classification?

In general the most popular approach is to calculate these measures for each class by comparing each class level to the remaining levels (i.e. a "one versus all" approach).

$\endgroup$
1
$\begingroup$

You simply have to merge the categories while calculating (e.g. $a$ and $\neg a$, where $\neg a$ is $b$ or $c$). True positive prediction for category $a$ is when $a$ was predicted and it was observed, true negative when it was not predicted and not observed, false negative when $a$ was observed but not predicted and false positive when it was predicted but not observed. You can find nice worked example of $3\times3$ confusion matrix on Wikipedia.

$\endgroup$
  • $\begingroup$ @Downvoter I would be happy to hear what exactly is wrong with this answer? $\endgroup$ – Tim Aug 15 '16 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.