2
$\begingroup$

I know that with k=1 a KNN lead to overfitting, this is because it follows the noisy data of the training sample and not generalize well on new input sample. But I am confused on how this happens, I understand the graphics but cant't figure out an example. For instance, why in the training data noise are followed and in test data not ? if someone can please give me an example, I'll be grateful.

Thanks to all :).

$\endgroup$

1 Answer 1

5
$\begingroup$

In the training data, each observation is its own 1-nearest neighbor and thus, the model predicts without error. Out-of-sample, this logic (fortunately) fails because the nearest neighbor is again selected from the training data.

$\endgroup$
1
  • $\begingroup$ Why the training data neighbor is its own? Hypotetically, I would delete the test data on training set to classify it, so there are no equal element. Can you please make an example, for instance, on classifying if a car 'x' is in class fiat or in class bmw. $\endgroup$ Commented Aug 29, 2022 at 8:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.