I want to understand the relationship (if any) between parameters and hyperparameters in a k-nearest-neighbor (KNN) model and how they relate to complexity.
A model which should classify handwritten numbers [0-9].
Dataset of N observation.
Below is my interpretation
Complexity in a KNN model is decided by the amount of features, 10 in this case, size of our dataset (N) and the value of K. If we have K=1, we will have a very complex model that will regard every datapoint, and effectively take into account N/1 = N parameters. Thereby a low K increases complexity by making the model regard every parameter.
Thinking about it as a linear problem, we would fit a very high degree polynomial when classifying.
If we however increase K, we simultaneously lower the amount of parameters the model have to take into account, thereby generating a less complex model. E.g K=2 would result in fitting a model on N/2 parameters.
Again thinking about it as a linear problem, we would end up with a lesser degree polynomial.
If we set K = N, we would have a model that's extremely simple that regards N/N = 1 parameter. In linear terms this would mean one single line that tries to separates our data.
I would like some confirmation on wheter I've understood this relationship between K, parameters and complexity correctly.