I'm watching a lecture on applying kNN to the digit recognition (MNIST dataset) problem. Let's say $n$ denotes the number of neighbors. This is a multiclass classification problem, so having an odd value of $n$ isn't enough to guarantee a majority.

My question is: what do we do in that case? Let's say $n=11$, we have a test sample image and its 11 nearest neighbors correspond to the training labels $(3, 3, 8, 8, 8, 9, 9, 9, 7, 7, 0)$. It's a very unlikely scenario but I just want theoretic clarification.

For example, do we just look at the labels with the highest count (in this case 8 and 9) and pick the one with the shortest average distance? pick the one with the shortest distance? etc.

What if the labels 3 and 7, despite having a lesser count than 8 and 9, have even shorter distances from the test sample than 8 and 9? (maybe the average distance is lesser or maybe even all the distances of 3 and 7 are less than each of the distances of 8 and 9).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.