2
$\begingroup$

My book on Linear regression: During prediction of $Y_0= \beta_0+\beta_1x_0+\epsilon_0$ from a new $x_0$ we have $var[\hat{Y}_0-Y_0]=var[\hat{Y}_0]+var[Y_0]$ since the covariance terms vanisces as $\epsilon_0$ is independent of $\epsilon_1, .., \epsilon_n$

hence it turns out $var[\hat{Y}_0-Y_0]=\sigma^2h_{00}+\sigma^2 = \sigma^2(1+h_{00})$.

however, later on, when calculating the variance of the residues it says: $var(R_i)=var(Y_i-\hat{Y}_i)=var[\hat{Y}_0]+var[Y_0]-2cov(Y_i, \hat{Y}_i)$

First of all: why $-2cov(Y_i, \hat{Y}_i)$ instead of $+2cov(Y_i, \hat{Y}_i)$?

Secondly, why the variance doesn't vanish here? The only thing that changes is that before we had a new observation, and afterwards we are studying the variance of the residuals (hence we are only using the original observations and the responses). But in any case we all know that $\epsilon_i \sim N(0, \sigma^2)$ IID, hence they are all independent of each other! For this reason surely the reason given by the book on why the covariance vanishes in the first case, is false.

$\endgroup$
1
  • $\begingroup$ you shouod really tag this as self-study --- but then yoo need to remove one other tag since there is a maximum of 5 $\endgroup$ Commented Apr 5, 2017 at 19:24

1 Answer 1

1
$\begingroup$

I finally managed to find out why.

When looking at residuals we know that $Y_i \in Y_1,..,Y_n$. But we also know that $\hat{\beta}_0 = \sum w_{0,i}Y_i$ and that $\hat{\beta}_1 = \sum w_{1,i}Y_i$ and therefore since we have $\hat{Y}_i=\hat{\beta}_0+\hat{\beta}_1x_i=\sum w_{0,i}Y_i + (\sum w_{1,i}Y_i)x_i$ i.e. $\hat{Y}_i$ and $Y_i$ are not independent, hence $cov \neq 0$.

However when doing prediction we have $Y_0 =\beta_0+\beta_1x_0+\epsilon_0$ and $\hat{Y}_0 = \hat{\beta}_0+\hat{\beta}_1x_0$. Now we know that $Y_0 \notin Y_1,..,Y_n$ and therefore we know that $Y_0$ and $\hat{Y}_0$ are independent, hence the covariance is zero

$\endgroup$
1
  • $\begingroup$ Although I still don't know why the minus.. $\endgroup$ Commented Apr 5, 2017 at 18:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.