3
$\begingroup$

The following is copy-pasted from a lecture note on machine learning...

k-NN classification can be realized in two ways:

  1. Selecting for the classified sample it's $k$ nearest neighbors for each class and computing than neighborhood radius $r_i$ for each of the classes. As the result of the classification, we select class $i$, for which its radius $r_i$ is the smallest.
  2. Selecting for the classified sample it's $k$ nearest neighbors in the training set (without taking class labels into account in this step). As the result of classification, we select class $i$ which is most numerous in the set of $k$ nearest neighbors of the classified sample.

Can anyone explain what points (1) and (2) talking about?

Are they talking about kNN for classification and kNN for regression, or something else?

$\endgroup$
4
  • $\begingroup$ Could you please post where you found those notes? $\endgroup$
    – Dave
    Jul 21, 2021 at 21:17
  • $\begingroup$ @Dave, No. The source is privacy protected. $\endgroup$
    – user366312
    Jul 21, 2021 at 21:57
  • $\begingroup$ Then what is $r_i?$ Is it how wide the circle around those $k$ points has to be to contain all $k$ plus the point we are trying to classify? $\endgroup$
    – Dave
    Jul 21, 2021 at 21:59
  • $\begingroup$ @Dave, $r_i$ is the neighborhood radius of $i$-th neighbor of a class, I guess. $\endgroup$
    – user366312
    Jul 21, 2021 at 22:08

1 Answer 1

1
$\begingroup$

Point #2 is the kNN I learned. It means that you look at the $k$ nearest points, say $7$, and make the classification based on those. If $5/7$ are $\text{cat}$ and $2/7$ are $\text{dog}$, you would predict a $\text{dog}$.$^{\dagger}$

Point #1 means that you select the $7$ nearest cats and the $7$ nearest dogs. I do not know what $r_i$ means, but perhaps that is defined elsewhere in the notes. Calculate $r_i$ for those $7$ cats and those $7$ dogs. Which $r_i$ is smallest of the $14?$ That class to which that belongs is the predicted class.

$^{\dagger}$Better yet, predict a $5/7$ probability of being a cat and a $2/7$ probability of being a dog.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.