Prove that $Var(\hat {Y_i})=\sigma^2h_{ii}$

Question

I just got a simple question.

In general linear model, we have $$\hat Y=HY$$ where $H=X(X^TX)^{-1}X^T$ and the residual $$E=Y-\hat Y.$$

Now I want to prove that $$Var(\hat {Y_i})=\sigma^2h_{ii}$$ where $h_{ii}$ is the $(i,i)$ entry in $H$. First I tried with $$Var(\hat {Y_i})=Cov(\hat{Y_i},\hat{Y_i})=Cov(Y_i-E_i,\hat{Y_i})=Cov(Y_i,\hat{Y_i})-Cov(E_i,\hat{Y_i}).$$

Now, $$Cov(Y_i,\hat{Y_i})=Cov(Y_i,\sum_{j=1}^nh_{ij}Y_j)=\sum_{j=1}^n h_{ij}Cov(Y_i,Y_j)=h_{ii}\sigma^2$$ under the model assumptions (independence of error components)

And I also proved that $$Cov(E_i,\hat{Y_i})=0.$$

Therefore I got the result.

However, when I try to directly prove from $$Var(\hat{Y_i})=Cov(\hat{Y_i},\hat{Y_i})=Cov(\sum_{j=1}^nh_{ij}Y_j,\sum_{k=1}^nh_{ik}Y_k)=\sum_{j=1}^n\sum_{k=1}^nh_{ij}h_{ik}Cov(Y_j,Y_k)={h_{ii}}^2\sigma^2$$ under the independence assumption, which is not the right answer.

I know my last step might be wrong, but 'm not sure about it. Could anyone help me?

Answer 1

Yes, you did make an error in the last step.

You are correct till,

$$Var(\hat{Y}_i) = \sum_{j=1}^{n} \sum_{k=1}^{n} h_{ij} h_{ik} Cov(Y_j, Y_k). $$

After this point note that $Cov(Y_j,Y_k) = 0$ when $j \ne y$. Thus, inside the first summation, I can replace the sum with $k = j$.

$$Var(\hat{Y}_i) = \sum_{j=1}^{n} h_{ij} h_{ij} Cov(Y_j, Y_j) = \sigma^2\sum_{j=1}^{n} h_{ij}^2 = \sigma^2h_{ii}, $$

where the last equality is because $H$ is idempotent and symmetric ($HH = H$).

---

As an aside, you made the problem more complicated than it should have been, by switching to Covariance. If you stick with variance and use independence of the $Y$s.

$$Var(\hat{Y}_i) = Var\left( \sum_{j=1}^{n}h_{ij}Y_j \right) = \sum_{j=1}^{n} h_{ij}^2 Var(Y_j) = \sigma^2h_{ii}. $$