Using the simple linear regression model: $ {y_i}= {\beta_0} + {\beta_1}x_i + \epsilon_i$, where E[$\epsilon_i$]=0 and var[$\epsilon_i] = \sigma^2$...

If $ \hat{y_i}= \hat{\beta_0} + \hat{\beta_1}x_i$ and

$ \hat{y_i^*}= \hat{\beta_0} + \hat{\beta_1}x_i^*$

(where $\hat{\beta_0}$ and $\hat{\beta_1}$ are ordinary least squares estimates of $\beta_0$ and $\beta_1$)

how would I go about deriving $\text{cov}$$(\hat{y_i}, \hat{y_i}^*)$?

Here is what I have been thinking:

$\text{cov}$$(\hat{y_i}, \hat{y_i}^*) = cov(\hat{\beta_0} + \hat{\beta_1}x_i, \hat{\beta_0} + \hat{\beta_1}x_i^*) = \beta_1^2 cov(x_i, x_i^*) = \beta_1^2[E[(x_i - E(x_i))(x_i^* - E(x_i^*))]] $

If the work that I've done is correct, is there any more steps that I can take?


The trick here is to realize that $x_i$ and $x_i^*$ are in this case constants.

$cov(\hat{y}_i, \hat{y}_i^*) = cov(\hat{\beta_0} + \hat{\beta}_1 x_i, \hat{\beta_0} + \hat{\beta}_1 x^*_i) = cov(\hat{\beta_0},\hat{\beta_0}) + cov(\hat{\beta_0},\hat{\beta_1}x_i^*) + cov( \hat{\beta}_1x_i,\hat{\beta_0}) +cov(\hat{\beta}_1x_i,\hat{\beta_1}x_i^*) = var(\hat{\beta_0}) + x_i^*cov(\hat{\beta_0},\hat{\beta_1}) + x_icov( \hat{\beta}_1,\hat{\beta_0}) + x_ix_i^*var(\hat{\beta}_1) = var(\hat{\beta_0}) + cov(\hat{\beta_0},\hat{\beta_1})(x_i+x_i^*) + x_ix_i^*var(\hat{\beta}_1) $.

Now the rest should be easy as the $var(\hat{\beta}_0),var(\hat{\beta}_1),cov(\hat{\beta}_0, \hat{\beta}_1)$ are all well known. See here the exact variance/covariance values: How to derive variance-covariance matrix of coefficients in linear regression


In response to your comment, it really is a matter of taste how far you take this from now on. I would proceed as follow:

$var(\hat{\beta_0}) + cov(\hat{\beta_0},\hat{\beta_1})(x_i+x_i^*) + x_ix_i^*var(\hat{\beta}_1) = \frac{\sigma^2 \sum x_i^2}{n\sum (x_i-\bar{X})^2} + (x_i+x_i^*)\frac{-\bar{X}\sigma^2}{\sum(x_i-\bar{X})^2}+ x_ix_i^*\frac{\sigma^2}{\sum(x_i - \bar{X})^2} = \frac{\sigma^2}{\sum(x_i-\bar{X})^2}(\frac{\sum x_i^2}{n}-\bar{X}(x_i+x_i^*)+x_ix_i^*)$

| cite | improve this answer | |
  • $\begingroup$ I apologize for starting a new thread, I will delete that one. Since: $var(\hat{\beta_0}) = \sigma^2(1/n + \bar{x}^2/S_{xx})$ and $var(\hat{\beta_1}) = \sigma^2/S_{xx}$ and $cov(\hat{\beta_0}, \hat{\beta_1}) = -\bar{x}(\sigma^2/S_{xx})$ Would it follow that: $\text{cov}$$(\hat{y_i}, \hat{y_i}^*) = \sigma^2(1/n + \bar{x}^2/S_{xx}) + (x_i^* + x_i)(-\bar{x}(\sigma^2/S_{xx})) + (x_ix_i^*)(\sigma^2/S_{xx}) = $ $(\sigma^2)[(1/n + \bar{x}^2/S_{xx}) + (x_i^* + x_i)(-\bar{x}/S_{xx}) + (x_ix_i^*/S_{xx})]$? And if the steps I took are correct, is there any further simplification to be done? $\endgroup$ – stats566 Apr 24 '14 at 22:07
  • $\begingroup$ It's ok, we were all new here at one point :-). Take a look at what I did. Hope that helps! And if it does, please do marke it as complete! :-). $\endgroup$ – user1357015 Apr 24 '14 at 22:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.