2
$\begingroup$

I am wondering what should happen when trying to compute leave-one-out cross-validation error when there are multiple 1-nearest-neighbours that is equidistant to the training point.

As an example, will the Leave-one-out cross-validation performed on the points below have an error of have an error of

  • Approach 1. 2/8 since the points at (2,6) and (6,2) will go by majority vote
  • Approach 2. 2/6 since the points at (2,6) and (6,2) will be not be classified and therefore will not be considered?

It seem to me that the answer is ambiguous and both could be possible solutions, albeit option 1 is likely to be most commonly chosen.

May I know if I use approach 2, stating that the two points will be unclassified, is incorrect? Example

$\endgroup$

1 Answer 1

1
$\begingroup$

There is no single correct tie breaking rule for knn algorithm. You can do majority voting, increase or decrease $k$ in cases of tie, do random sampling etc. The choice of leaving the points unclassified will depend on your way of handling ties. It is not incorrect, it's probably just unusual.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.