In ImageNet classification papers top-1 and top-5 error rates are important units for measuring the success of some solutions, but what are those error rates?

In ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky et al. every solution based on one single CNN (page 7) has no top-5 error rates while the ones with 5 and 7 CNNs have (and also the error rate for 7 CNNs are better than for 5 CNNs).

Does this mean top-1 error rate is the best single error rate for one single CNN?

Is the top-5 error rate simply the accumulated error rate of five CNNs?

up vote 35 down vote accepted

[...] where the top-5 error rate is the fraction of test images for which the correct label is not among the five labels considered most probable by the mode.

First, you make a prediction using the CNN and obtain the predicted class multinomial distribution ($\sum p_{class} = 1$).

Now, in the case of top-1 score, you check if the top class (the one having the highest probability) is the same as the target label.

In the case of top-5 score, you check if the target label is one of your top 5 predictions (the 5 ones with the highest probabilities).

In both cases, the top score is computed as the times a predicted label matched the target label, divided by the number of data-points evaluated.

Finally, when 5-CNNs are used, you first average their predictions and follow the same procedure for calculating the top-1 and top-5 scores.

Your classifier gives you a probability for each class. Lets say we had only "cat", "dog", "house", "mouse" as classes (in this order). Then the classifier gives somehting like

0.1; 0.2; 0.0; 0.7

as a result. The Top-1 class is "mouse". The top-2 classes are {mouse, dog}. If the correct class was "dog", it would be counted as "correct" for the Top-2 accuracy, but as wrong for the Top-1 accuracy.

Hence, in a classification problem with $k$ possible classes, every classifier has 100% top-$k$ accuracy. The "normal" accuracy is top-1.

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.