I understood that KNN (K-Nearest-Neighbors) was non-parametric, after reading the beginning of the wikipedia article here:

In pattern recognition, the $k$-Nearest Neighbors algorithm (or $k$-NN for short) is a non-parametric method used for...

But, then later in the article it talks of estimating the "parameters"??

The best choice of $k$ depends upon the data; generally, larger values of $k$ reduce the effect of noise on the classification,[5] but make boundaries between classes less distinct.

Am I missing the difference between parameters and hyperparameters? Thanks.

• There are a number of similar questions on the distinction between parametric and non-parametric. (1) stats.stackexchange.com/questions/46588/… (2) stats.stackexchange.com/questions/103833/… (3) stats.stackexchange.com/questions/185909/…
– Sycorax
Dec 30, 2015 at 17:35
• short answer: non-parametric can been interpreted as unfixed number of parameters (or all possible solutions cannot be defined by a finite number of parameters) . So because the final number of parameters is determined by the data, it can be considered non-parametric. Dec 30, 2015 at 17:55
• Thanks. I'm interested in this for the context of machine learning, (random forest, kNN, SVM, etc.) So, two clarifications: fixed for what - the function that your ML algorithm produces for predicting new labels/target variables? Dec 30, 2015 at 22:36
• @Candic3: the function that is produced is a function of various parameters. The question is: before seeing your data, do you know how many parameters are necessary to describe this function (or moreover, the model that the function uses to produce new predictions)? Dec 31, 2015 at 0:28