I just got a simple question.

In general linear model, we have $$\hat Y=HY$$ where $H=X(X^TX)^{-1}X^T$ and the residual $$E=Y-\hat Y.$$

Now I want to prove that $$Var(\hat {Y_i})=\sigma^2h_{ii}$$ where $h_{ii}$ is the $(i,i)$ entry in $H$. First I tried with $$Var(\hat {Y_i})=Cov(\hat{Y_i},\hat{Y_i})=Cov(Y_i-E_i,\hat{Y_i})=Cov(Y_i,\hat{Y_i})-Cov(E_i,\hat{Y_i}).$$

Now, $$Cov(Y_i,\hat{Y_i})=Cov(Y_i,\sum_{j=1}^nh_{ij}Y_j)=\sum_{j=1}^n h_{ij}Cov(Y_i,Y_j)=h_{ii}\sigma^2$$ under the model assumptions (independence of error components)

And I also proved that $$Cov(E_i,\hat{Y_i})=0.$$

Therefore I got the result.

However, when I try to directly prove from $$Var(\hat{Y_i})=Cov(\hat{Y_i},\hat{Y_i})=Cov(\sum_{j=1}^nh_{ij}Y_j,\sum_{k=1}^nh_{ik}Y_k)=\sum_{j=1}^n\sum_{k=1}^nh_{ij}h_{ik}Cov(Y_j,Y_k)={h_{ii}}^2\sigma^2$$ under the independence assumption, which is not the right answer.

I know my last step might be wrong, but 'm not sure about it. Could anyone help me?


1 Answer 1


Yes, you did make an error in the last step.

You are correct till,

$$Var(\hat{Y}_i) = \sum_{j=1}^{n} \sum_{k=1}^{n} h_{ij} h_{ik} Cov(Y_j, Y_k). $$

After this point note that $Cov(Y_j,Y_k) = 0$ when $j \ne y$. Thus, inside the first summation, I can replace the sum with $k = j$.

$$Var(\hat{Y}_i) = \sum_{j=1}^{n} h_{ij} h_{ij} Cov(Y_j, Y_j) = \sigma^2\sum_{j=1}^{n} h_{ij}^2 = \sigma^2h_{ii}, $$

where the last equality is because $H$ is idempotent and symmetric ($HH = H$).

As an aside, you made the problem more complicated than it should have been, by switching to Covariance. If you stick with variance and use independence of the $Y$s.

$$Var(\hat{Y}_i) = Var\left( \sum_{j=1}^{n}h_{ij}Y_j \right) = \sum_{j=1}^{n} h_{ij}^2 Var(Y_j) = \sigma^2h_{ii}. $$


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.