# Covariance Question

I am asked to show $\operatorname{Cov}(\hat{Y},r)=0$ in the least squares regression setting. I have tried 2 different approaches and have got stuck on both approaches. Here are my attempts

. Any help to get me in the right direction is much appreciated. Thanks in advance. $\hat{Y}$ is the fitted values, $\beta$ is the parameter vector, and $\epsilon$ the error.

• can you define your notation? – Taylor Sep 17 '17 at 19:17

If you want to use your second approach, you just need to fix a few things.

\begin{align} &\mathbb{E}[(\hat{y} - X \beta)r^\top] \\ &= \mathbb{E}[\hat{y} r^\top - X \beta r^\top]\\ &= \mathbb{E}[\hat{y}(y - \hat{y})^\top] - X\beta \mathbb{E}[r^\top] \\ &= \mathbb{E}[\hat{y}(y - \hat{y})^\top] \\ &= \mathbb{E}[Hy(y - Hy)^\top] \\ &= H \mathbb{E}[yy^\top] (I - H) \\ &= H (\sigma^2 I) (I - H) & \text{assuming $y \sim N(X\beta, \sigma^2 I)$} \\ &= \sigma^2 H(I-H) \\ &= 0, \end{align} where $H = X(X^\top X)^{-1}X^\top$ is the projection onto the column space of $X$.

Similar approach:

\begin{align} \hat{y} &= H y \\ r &= y - \hat{y} = (I - H)y \\ \mathbb{E} r &= 0, \end{align}

Plugging all of this into your starting point yields \begin{align} &\mathbb{E}[(\hat{y} - \mathbb{E} \hat{y})(r - \mathbb{E} r)^\top] \\ &= \mathbb{E}[H(y - \mathbb{E}[y])y^\top(I - H)^\top] \\ &= H\mathbb{E}[\epsilon (X\beta + \epsilon)^\top] (I - H) \\ &= H \mathbb{E}[\epsilon \epsilon^\top] (I - H) \\ &= \sigma^2 H(I-H) & \text{assuming $\epsilon \sim N(0, \sigma^2 I)$} \\ &= 0. \end{align}

• Thank you very much. When expanding the transpose into the brackets does the transpose apply to H or just Y? – TimHorton Sep 17 '17 at 20:42
• @Kappa $H$ is symmetric so it does not end up mattering. – angryavian Sep 17 '17 at 21:21