0
$\begingroup$

I'm doing research on image classification, and I don't understand the difference between Top-1 Accuracy and Recall. Are they the same?

I found a link here What is the definition of Top-n accuracy? that explains what top-n accuracy is, which is the total number of correct guesses divided by the total number of images.

But isn't that recall? Recall is defined as the number of correct guesses divided by the total number of images

Given these laymen's terms definitions:

`TP` = Making a correct guess
`TN` = Not making a wrong guess
`FP` = Guessing the wrong class
`FN` = Missing the correct class
`FP` + `TP` = total number of guesses
`FN` + `TP` = total number of images

Accuracy is defined as

(TN + TP) / (TN + TP + FN + FP)

Recall is defined as

TP / (FN + TP)

But Top-1 Accuracy is defined as

TP / (FN + TP) where we only look at the 1st guess, ignoring others

Similarly Top-N Accuracy is defined as

TP / (FN + TP) where we only look at the top N guesses, ignoring others

Have I misunderstood what Top-1 Accuracy is? Shouldn't Top-1 Accuracy be Accuracy as it's defined (TN + TP) / (TN + TP + FN + FP)? Otherwise it should be called Top-1 Recall

Is Recall equal to Top-1 Accuracy and Recall is just a subset of Top-N Accuracy where N equals 1?

$\endgroup$
1
  • $\begingroup$ Similarly could we measure Top-N Precision, (TP / total number of guesses)? $\endgroup$
    – JJrodny
    Jul 15, 2020 at 15:49

1 Answer 1

1
$\begingroup$

Some of your definitions are not correct. Let's start from the very beginning. Imagine there are two classes of images: class 0 (negative) and class 1 (positive). For example, the positive class can consist of images of cats, and the negative class — of images without cats. Then we build a binary classifier that detects whether the image contains a cat (positive decision) or not (negative decision). We want to describe its performance. A false positive is when the classifier predicts the positive class (it detects a cat in the image), but in reality there is none (the image is of the negative class). Conversely, a false negative is when the classifier incorrectly predicts the negative class (the cat is in the image, but it doesn't get detected). We evaluate the classifier on all the images, which means that the number of guesses is the same as the number of images, and both equal TP + TN + FP + FN (where ‘TP’ denotes the number of true positive decisions, and so on).

Accuracy is the fraction of images in which the classifier has guessed the true class (whichever it is) correctly. For a binary classifier it is computed as $\frac{\text{TP} + \text{TN}}{\text{TP} + \text{TN} + \text{FP} + \text{FN}}$. The top-$n$ accuracy is for a multiclass classifier, and this formula is not applicable anymore. But the textual definition still holds. Recall for a binary classifier is $\frac{\text{TP}}{\text{TP} + \text{FN}}$. It shows in what fraction, out of all images with cats, cats have been detected. I'm not sure I've ever seen the recall used with multiclass classification, but if one were to generalize it, it would make sense to compute it for a single class: Among images containing cats, in what fraction the ‘cat’ class is found among the top $n$ classes reported by the detector? You can see that this is different from the accuracy.

$\endgroup$
4
  • $\begingroup$ I guess I'm more interested in multi-class classification. For my laymen's terms I simplified too much, I meant: --- TP: correctly guessing cat on a cat image - TN: correctly not guessing cat on a non-cat image - FP: incorrectly guessing cat on a non-cat image - FN: incorrectly guessing no cat on a cat image (incorrectly guessing any other class than cat) --- And so it seems to me that Recall (as you correctly stated it) is the same as the definition of Top-1 Accuracy in multi-class image classification, right? Or is it different? $\endgroup$
    – JJrodny
    Jul 21, 2020 at 20:43
  • $\begingroup$ Also, a confusing point, but as I understand it: Top-1 Accuracy is different from Accuracy. Accuracy is as you said (TP+TN)/(TP+TN+FP+FN), Whereas Top-1 Accuracy as described in that post I linked is (TP)/(TP+FN) $\endgroup$
    – JJrodny
    Jul 21, 2020 at 21:56
  • $\begingroup$ @JJrodny Accuracy is always about all classes. If you have 10 classes, you would count as correct guesses all images of class 1 where the classifier reported class 1 (as the top one, among top 5, etc), all images of class 2 where class 2 was reported, and so on until class 10. $\endgroup$ Jul 22, 2020 at 15:10
  • $\begingroup$ But I can calculate the accuracy of one class in a 10 class dataset, can't I? I can get the TN, TP, FP, FN for each class separately, calculate the accuracy for each class individually as in that formula, (and then if I so choose I can get the average across-class accuracy by averaging each class's accuracy). If I instead combine TNs, TPs, FNs, FPs, across classes and then put them in the formula that measurement would favor the larger unbalanced classes (A model that only guesses dogs on a dataset of 500 dogs, 2 cats = 1000/1004 = 96.6% accuracy (bad) vs (500/500 + 0/2)/2 = 50% accuracy) $\endgroup$
    – JJrodny
    Jul 22, 2020 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.