I am having problems calculating the variance of a residual in an SLR setting,
ie $\text{var}$$(y_i- \hat{y_i})$. Here is what I have thus far.

If $ \hat{y_i}= \hat{\beta_0} + \hat{\beta_1}x_i$ and

$ \ y_i= \beta_0 + \beta_1x_i$

(where $\hat{\beta_0}$ and $\hat{\beta_1}$ are ordinary least squares estimates of $\beta_0$ and $\beta_1$).

$ \text{var}$$(e_i)=\text{var}$$(y_i- \hat{y_i})$
$ \ =\text{var}$$(y_i)+\text{var}$$(\hat{y_i})-2*\text{cov}$$(y_i, \hat{y_i}) $
$ \ =\sigma^2 + \sigma^2( \frac{1}{\ n} +\frac{(x_i-\bar{x})^2\sigma^2}{\sum(xi-\bar{x})^2})-? $

Now what is screwing me up is the covariance term. When I was studying prediction intervals we assume that $ \hat{y_i} $ has no bearing on $\ y_i $. I was told by classmate that this is not the case and that $ \text{cov}$$(y_i, \hat{y_i})=\text{var}$$(y_i) $. Which option is correct, if any? And what is the difference between the variance we find for prediction intervals and this one?

Thanks in advance!


When doing prediction intervals, you're doing that calculation for an observation that's not used in the estimation, so (by the regression assumptions themselves) $\hat{y}_i$ has no bearing on $y_i$.

With residuals the observation is used in the estimation, so the two are dependent.

$\text{cov}(y_i, \hat{y}_i) = \text{cov}(\hat{y}_i+y_i-\hat{y}_i, \hat{y}_i) = \text{cov}(\hat{y}_i, \hat{y}_i) +\text{cov}(y_i-\hat{y_i}, \hat{y}_i) = \text{var}(\hat{y}_i)+0$

For additional details, see here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.