Gaussian kernel estimator as Nadaraya-Watson estimator?

Question

I'm working on a problem from "The Elements of Statistical Learning" (prob. 6.8):

Suppose that for continuous response $Y$ and predictor $X$, we model the joint density of $X, Y$ using a multivariate Gaussian kernel estimator. Note that the kernel in this case would be the product kernel $\phi_{\lambda}(X) \phi_{\lambda}(Y)$.

(a) Show that the conditional mean $E(Y|X)$ derived from this estimate is a Nadaraya-Watson estimator.

(b) Extend this result to classification by providing a suitable kernel for the estimation of the joint distribution of a continuous $X$ and discrete $Y$.

I know that the Nadaraya-Watson estimator is just the weighted average (equation 2.41 and 6.2 in ESL):

$$\hat f (x_0) = \frac{\sum_{i=0}^N K_{\lambda}(x_0, x_i) y_i}{\sum_{i=0}^N K_{\lambda}(x_0, x_i)}$$

Where $K$ in this case would be the multivariate Gaussian kernel function.

I can think about how to extend this to a classification problem, but am not sure how to approach the first part of this question.

Any pointers would be greatly appreciated!

Answer 1

The conditional mean is defined by:

$$E(Y|X)\equiv\int y f(y|x) dy$$

Where $f(Y|X)$ is the conditional density. Using the product rule, you can show:

$$f(y|x)=\frac{f(y,x)}{f(x)}$$

Substituting this back into the integral you get

$$E(Y|X)\equiv\frac{\int y f(y,x) dy}{f(x)}$$

Which is of the form you seek, if you use the kernel density estimator.