0
$\begingroup$

I understood that KNN (K-Nearest-Neighbors) was non-parametric, after reading the beginning of the wikipedia article here:

In pattern recognition, the $k$-Nearest Neighbors algorithm (or $k$-NN for short) is a non-parametric method used for...

But, then later in the article it talks of estimating the "parameters"??

The best choice of $k$ depends upon the data; generally, larger values of $k$ reduce the effect of noise on the classification,[5] but make boundaries between classes less distinct.

Am I missing the difference between parameters and hyperparameters? Thanks.

|cite|improve this question
$\endgroup$
  • 2
    $\begingroup$ There are a number of similar questions on the distinction between parametric and non-parametric. (1) stats.stackexchange.com/questions/46588/… (2) stats.stackexchange.com/questions/103833/… (3) stats.stackexchange.com/questions/185909/… $\endgroup$ – Sycorax Dec 30 '15 at 17:35
  • 2
    $\begingroup$ short answer: non-parametric can been interpreted as unfixed number of parameters (or all possible solutions cannot be defined by a finite number of parameters) . So because the final number of parameters is determined by the data, it can be considered non-parametric. $\endgroup$ – Cliff AB Dec 30 '15 at 17:55
  • $\begingroup$ Thanks. I'm interested in this for the context of machine learning, (random forest, kNN, SVM, etc.) So, two clarifications: fixed for what - the function that your ML algorithm produces for predicting new labels/target variables? $\endgroup$ – Candic3 Dec 30 '15 at 22:36
  • 1
    $\begingroup$ @Candic3: the function that is produced is a function of various parameters. The question is: before seeing your data, do you know how many parameters are necessary to describe this function (or moreover, the model that the function uses to produce new predictions)? $\endgroup$ – Cliff AB Dec 31 '15 at 0:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.