0
$\begingroup$

Question: Estimate the error rate of the one-nearest-neighbor (1-NN) classifier for this problem using leave-one-out cross validation. (That is, using S-fold cross validation with S equal to the number of training cases, in which each training case is predicted using all the other training cases.)

enter image description here

Could someone walk me through the logic in this question? It is supposed to help us understand the intuition of how KNN works.

Thank you!

$\endgroup$
0
$\begingroup$

I think you guessed it right- this is usually an example to show how overfitting/underfitting varies with choice of k.

If you choose k=1, the algorithm will pick the neighbor as the point itself and have a extremely overfit decision boundary. When you do a leave one out cv the error you will thus obtain will be high.

If you choose k=(no of data points), the decision boundary will be the average line for all your data points (extreme underfitting). In this case too you will obtain a high error.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.