I have the following question in my textbook:
One of the drawbacks of the nearest-neighbour algorithm is that we must retain all of the training data. Describe a situation where a training point can be removed without affecting the resulting 1-NN classification for any test point in the input space.
One such situation would be if there is only 1 class? Then all of the future test points would be classified as that class.
I can't imagine another situation since from my understanding, for the 1-NN to work, there needs to be at least some datapoints which it would "compare similarity" (measure the distance) with and decide whether it would be some class.
What would other situations look like that aren't what I thought of?