5
$\begingroup$

OK, I could go through the code to figure this out but I feel something Googleable doesn't hurt.

When I'm using a kNN classifier with (inverse) distance weighting, how does it handle cases whereby the distance between the prediction input and (m)any of the k nearest training records is zero? What if some of them aren't?

Improve this question
$\endgroup$
5
$\begingroup$

KNN classifier in scikit-learn uses _get_weights method in sklearn.neighbors.base library. The inverse weighting is achieved when 'distance' is given as weights paremeter. You can also call this function directly by giving your distances as input. The weight is $w=\frac{1}{d}$, but surprisingly, when $d$ is $0$, the weight is always set to $1$. In the code, it does an np.isinf check and when a weight is infinite, it is set to the boolean value produced by np.isinf.

Improve this answer
$\endgroup$
  • 1
    $\begingroup$ Thanks. Mmh. I was hoping that in a mixed case where say first nearest neighbour has distance 0, it would trump any other neighbour with distance >0. Setting it to weight=1 doesn't seem right. What would the "correct" approach be? Simple voting among all neighbours with distance=0 and discarding all other neihbours? $\endgroup$ – faph Nov 25 '18 at 17:21
  • 1
    $\begingroup$ I'd at least set the weight to a large value, using some epsilon, or maybe discard that sample. But, I wouldn't do weight = 1. I don't know why they choose such a solution. $\endgroup$ – gunes Nov 26 '18 at 15:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.