# Why effective number of parameters in K nearest neighbor is N/k?

Bellow is my deduction:

According to the definition of k-NN fit, we have $$\hat{Y}(x) = \frac{1}{k} \sum_{x_i \in N_k(x)}^{N}= \frac{1}{k}diag(a_1, a_2,..., a_N)y$$ where $$N_k(x)$$ is the neighborhood of $$x$$ defined by the k closest points $$x_i$$ in the training sample, and $$diag(a_1, a_2,..., a_N)$$ is a diagonal matrix, if $$x_i \in N_k(x)$$, $$a_i=1$$, else $$a_i=0$$.

Hence, the effective degrees-of-freedom $$df({S}) = trace({S})=trace(\frac{1}{k}diag(a_1, a_2,..., a_N))=\frac{trace(diag(a_1, a_2,..., a_N))}{K}=1$$

What's wrong with my deduction?

• If $Y$ is multivariate normal, $\hat{Y}(x)$ is a linear combination of $k$-normal random variates. If the variance is unknown, the degrees of freedom depend on how you estimate the variance. Are you using $\sum_{i=1}^n (Y_i - \hat{Y}(x))^2 = SSYY$? Commented Nov 25, 2018 at 14:00

Sorry, I misunderstood the definition of $$S$$. Maybe below is a right deduction.
According to the definition of k-NN fit, we have $$\hat{Y}(x) = \frac{1}{k} \sum_{x_i \in N_k(x)}^{N}= \frac{1}{k}\boldsymbol{I}_N\boldsymbol{y}$$
where $$N_k(x)$$ is the neighborhood of $$x$$ defined by the k closest points $$x_i$$ in the training sample, $$\boldsymbol{I_N}$$ is a $$1*N$$ row vector and the i-th element $$\boldsymbol{I_N}(i)=1$$ if $$x_i \in N_k(x)$$, else $$\boldsymbol{I_N}(i)=0$$
Suppose we stack the outcomes $$y_1, y_2, ..., y_N$$ into a vector $$\boldsymbol{y}$$ , and similarly for the predictions $$\boldsymbol{\hat{y}}$$. Then the fitting method is one for which we can write $$\boldsymbol{\hat{y}}= \boldsymbol{Sy}$$ where $$\boldsymbol{S}$$ is an $$N*N$$ matrix depending on the input vectors $$x_i$$ but not on the $$y_i$$.
Hence, the effective degrees-of-freedom is $$df(\boldsymbol{S}) = trace(\boldsymbol{S})=trace(\frac{1}{k}\boldsymbol{I}_{N*N}) \approx \frac{N}{K}$$