# Show $\hat{f}(x) \to E[Y|X=x]$ elements of statistical learning

I was reading elements of statistical learning and it mentions let $$\hat{f}(x)=Ave(y_i|x_i \in N_k(x))$$ where $$N_k(x)$$ is the neighborhood containing the k points closest to x .

Then it says "under mild regularity conditions on the joint probability distribution $$Pr(X, Y )$$, one can show that as $$N, k \to \infty$$ such that $$k/N \to 0$$, $$\hat{f}(x) \to E(Y |X = x)$$.

Can someone guide how to begin proving this statement and what mild regularity conditions are required?

• Might want to check stats.stackexchange.com/questions/96044/… for pointers in the answer. Oct 26, 2023 at 19:32
• Although the details of "one can show that" is definitely not easy, the first step of approaching the problem is rewrite $\hat{f}(x)$ as the weighted sum of $Y_i$, i.e., $\hat{f}(x) = \sum_{i = 1}^n W_{n, i}Y_i$. One thing the author of ESL failed to make this statement sufficiently precise is that they did not specify the convergence mode: the conditions and methods used to prove $\hat{f}(x) \to_p f(x)$, $\hat{f}(x) \to_{a.s.} f(x)$, and $\hat{f}(x) \to_{L^p} f(x)$ are different, where $f(x) = E[Y|X = x]$ (if you are a mathematician, ESL is pretty painful to read...). Oct 27, 2023 at 0:55