0
$\begingroup$

I am asked to show $\operatorname{Cov}(\hat{Y},r)=0$ in the least squares regression setting. I have tried 2 different approaches and have got stuck on both approaches. Here are my attempts

enter image description here

. Any help to get me in the right direction is much appreciated. Thanks in advance. $\hat{Y}$ is the fitted values, $\beta$ is the parameter vector, and $\epsilon$ the error.

$\endgroup$
  • $\begingroup$ can you define your notation? $\endgroup$ – Taylor Sep 17 '17 at 19:17
0
$\begingroup$

If you want to use your second approach, you just need to fix a few things.

\begin{align} &\mathbb{E}[(\hat{y} - X \beta)r^\top] \\ &= \mathbb{E}[\hat{y} r^\top - X \beta r^\top]\\ &= \mathbb{E}[\hat{y}(y - \hat{y})^\top] - X\beta \mathbb{E}[r^\top] \\ &= \mathbb{E}[\hat{y}(y - \hat{y})^\top] \\ &= \mathbb{E}[Hy(y - Hy)^\top] \\ &= H \mathbb{E}[yy^\top] (I - H) \\ &= H (\sigma^2 I) (I - H) & \text{assuming $y \sim N(X\beta, \sigma^2 I)$} \\ &= \sigma^2 H(I-H) \\ &= 0, \end{align} where $H = X(X^\top X)^{-1}X^\top$ is the projection onto the column space of $X$.


Similar approach:

\begin{align} \hat{y} &= H y \\ r &= y - \hat{y} = (I - H)y \\ \mathbb{E} r &= 0, \end{align}

Plugging all of this into your starting point yields \begin{align} &\mathbb{E}[(\hat{y} - \mathbb{E} \hat{y})(r - \mathbb{E} r)^\top] \\ &= \mathbb{E}[H(y - \mathbb{E}[y])y^\top(I - H)^\top] \\ &= H\mathbb{E}[\epsilon (X\beta + \epsilon)^\top] (I - H) \\ &= H \mathbb{E}[\epsilon \epsilon^\top] (I - H) \\ &= \sigma^2 H(I-H) & \text{assuming $\epsilon \sim N(0, \sigma^2 I)$} \\ &= 0. \end{align}

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thank you very much. When expanding the transpose into the brackets does the transpose apply to H or just Y? $\endgroup$ – TimHorton Sep 17 '17 at 20:42
  • $\begingroup$ @Kappa $H$ is symmetric so it does not end up mattering. $\endgroup$ – angryavian Sep 17 '17 at 21:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.