Let $(x_i,y_i)_{1\leq i\leq n}$ some dataset. I want to estimate the conditional expectation $E[Y\mid X=x]$ and the conditional variance $V[Y\mid X=x]$.

I used Nadaraya-Watson's estimator to estimate the conditional expectation: $$\hat{E}[Y\mid X=x]=\frac{\sum_{i=1}^ny_iK\left(\frac{x-x_i}{h}\right)}{\sum_{i=1}^nK\left(\frac{x-x_i}{h}\right)}$$ In fact I just use the ksmooth function in R.

Now let $z_i$ the squared residual: $z_i = \left(y_i - \hat{E}[Y\mid X=x_i]\right)^2$. Then I use Nadaraya-Watson's estimator once again to get an estimation of the conditional variance: $$\hat{V}[Y\mid X=x]=\frac{\sum_{i=1}^nz_iK\left(\frac{x-x_i}{h}\right)}{\sum_{i=1}^nK\left(\frac{x-x_i}{h}\right)}$$

So this is some kind of two-step estimation, which bothers me a little. Is this a good way to estimate the conditional variance? If not, how to do this properly (and not paramatrically)?

  • $\begingroup$ Use a bootstrap? $\endgroup$ – Repmat Sep 5 '16 at 18:01
  • $\begingroup$ It seems reasonable to me. Consider, are you bothered by the comparable two-step estimation in the simple case of the overall (marginal) $y$ variance? (That is simply the limit of uniform weights, i.e. as your bandwidth $h$ goes to infinity.) $\endgroup$ – GeoMatt22 Sep 5 '16 at 18:43

You could simply use the fact that $V(Y) = \mathbb{E}[Y^2] - \mathbb{E}[Y]^2$, estimate the conditional expectations of $Y^2$ and $Y$ separately, then estimate the conditional variance by: $$\hat{V}(Y|X) = \hat{\mathbb{E}}(Y^2|X) - \hat{\mathbb{E}}(Y|X)^2$$ No double smoothing involved this way.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.