1
$\begingroup$

Python offers two nearest neighbor regressions: radius nearest neighbor and k-nearest neighbor. I'm trying to figure out a few things: 1. Under which circumstances would each be preferable? 2. How do you approach setting the optimal radius or k value?

For reference, I'm working with a relatively sparse data set with a uniform geometry. As time passes, that dataset will get less sparse, but will continue to have a relatively uniform geometry.

Thanks for any help.

|cite|improve this question
$\endgroup$
-1
$\begingroup$
  1. Under which circumstances would each be preferable?

You should try each on your dataset to be sure.

  1. How do you approach setting the optimal radius or k value?

You optimize this value to give you the best results on the training set. Then, you report the performance it gives you on the test set.

For example, train on a 90% random partition of your data. Use the remaining 10% to test the value you have chosen. You can also use 5 folds cross validation if you want a better way to assess the performance of your classifier.

|cite|improve this answer
$\endgroup$
  • 1
    $\begingroup$ This answer to (2) is incorrect and will lead to overfitting. If you optimize $k$ or the radius on the training set, the optimal choice of $k$ will be 1, and the optimal radius will be zero (or any value smaller than the minimum distance to the next-nearest neighbor). This is because each point in the training set is its own nearest neighbor, and outputting its corresponding target value will give zero error on the training set. This will probably not generalize well to new data. The proper procedure is to select $k$ or the radius using cross validation or an independent validation set. $\endgroup$ – user20160 Nov 11 '18 at 6:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.