Using the simple linear regression model: $ {y_i}= {\beta_0} + {\beta_1}x_i + \epsilon_i$, where E[$\epsilon_i$]=0 and var[$\epsilon_i] = \sigma^2$...

If $ \hat{y_i}= \hat{\beta_0} + \hat{\beta_1}x_i$ and

$ \hat{y_i^*}= \hat{\beta_0} + \hat{\beta_1}x_i^*$

(where $\hat{\beta_0}$ and $\hat{\beta_1}$ are ordinary least squares estimates of $\beta_0$ and $\beta_1$)

how would I go about deriving $\text{cov}$$(\hat{y_i}, \hat{y_i}^*)$?

Here is what I have been thinking:

$\text{cov}$$(\hat{y_i}, \hat{y_i}^*) = cov(\hat{\beta_0} + \hat{\beta_1}x_i, \hat{\beta_0} + \hat{\beta_1}x_i^*) = \beta_1^2 cov(x_i, x_i^*) = \beta_1^2[E[(x_i - E(x_i))(x_i^* - E(x_i^*))]] $

If the work that I've done is correct, is there any more steps that I can take?


1 Answer 1


The trick here is to realize that $x_i$ and $x_i^*$ are in this case constants.

$cov(\hat{y}_i, \hat{y}_i^*) = cov(\hat{\beta_0} + \hat{\beta}_1 x_i, \hat{\beta_0} + \hat{\beta}_1 x^*_i) = cov(\hat{\beta_0},\hat{\beta_0}) + cov(\hat{\beta_0},\hat{\beta_1}x_i^*) + cov( \hat{\beta}_1x_i,\hat{\beta_0}) +cov(\hat{\beta}_1x_i,\hat{\beta_1}x_i^*) = var(\hat{\beta_0}) + x_i^*cov(\hat{\beta_0},\hat{\beta_1}) + x_icov( \hat{\beta}_1,\hat{\beta_0}) + x_ix_i^*var(\hat{\beta}_1) = var(\hat{\beta_0}) + cov(\hat{\beta_0},\hat{\beta_1})(x_i+x_i^*) + x_ix_i^*var(\hat{\beta}_1) $.

Now the rest should be easy as the $var(\hat{\beta}_0),var(\hat{\beta}_1),cov(\hat{\beta}_0, \hat{\beta}_1)$ are all well known. See here the exact variance/covariance values: How to derive variance-covariance matrix of coefficients in linear regression


In response to your comment, it really is a matter of taste how far you take this from now on. I would proceed as follow:

$var(\hat{\beta_0}) + cov(\hat{\beta_0},\hat{\beta_1})(x_i+x_i^*) + x_ix_i^*var(\hat{\beta}_1) = \frac{\sigma^2 \sum x_i^2}{n\sum (x_i-\bar{X})^2} + (x_i+x_i^*)\frac{-\bar{X}\sigma^2}{\sum(x_i-\bar{X})^2}+ x_ix_i^*\frac{\sigma^2}{\sum(x_i - \bar{X})^2} = \frac{\sigma^2}{\sum(x_i-\bar{X})^2}(\frac{\sum x_i^2}{n}-\bar{X}(x_i+x_i^*)+x_ix_i^*)$

  • $\begingroup$ I apologize for starting a new thread, I will delete that one. Since: $var(\hat{\beta_0}) = \sigma^2(1/n + \bar{x}^2/S_{xx})$ and $var(\hat{\beta_1}) = \sigma^2/S_{xx}$ and $cov(\hat{\beta_0}, \hat{\beta_1}) = -\bar{x}(\sigma^2/S_{xx})$ Would it follow that: $\text{cov}$$(\hat{y_i}, \hat{y_i}^*) = \sigma^2(1/n + \bar{x}^2/S_{xx}) + (x_i^* + x_i)(-\bar{x}(\sigma^2/S_{xx})) + (x_ix_i^*)(\sigma^2/S_{xx}) = $ $(\sigma^2)[(1/n + \bar{x}^2/S_{xx}) + (x_i^* + x_i)(-\bar{x}/S_{xx}) + (x_ix_i^*/S_{xx})]$? And if the steps I took are correct, is there any further simplification to be done? $\endgroup$
    – stats566
    Commented Apr 24, 2014 at 22:07
  • $\begingroup$ It's ok, we were all new here at one point :-). Take a look at what I did. Hope that helps! And if it does, please do marke it as complete! :-). $\endgroup$ Commented Apr 24, 2014 at 22:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.