3
$\begingroup$

I am at a loss how might one prove $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$. Note that $\hat{y}_h$ = $b_0 + b_1X_h$ which is a regression line estimate at some given $X_h$.

Moreover, what is the second term actually saying? I understand the first is just the variance divided by the degrees of freedom but the second is more complex it seems.

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ please add the self-study tag $\endgroup$
    – Glen_b
    Commented May 29, 2013 at 0:52

1 Answer 1

Reset to default
1
$\begingroup$

Here's an outline:

  • Write down the formula for $\hat{y}_h$.

  • Take the variance of it.

The second involves rather similar steps to your earlier variance question.

The second term is related to the effect of the uncertainty in the slope estimate on the uncertainty in the fitted value; this gets bigger the further you get from the mean $x$.

I am quite rusty on how to take the variance of multiple terms like this

Just keep using basic properties of things you know about, like:

$\text{Var}(X+Y) = \text{Var}(X) + \text{Var}(Y) + 2\, \text{Cov}(X,Y)$

$\text{Var}(aX) = a^2\text{Var}(X)$

$ \text{Cov}(aX,bY)= ab\,\text{Cov}(X,Y)$

and so on

Take advantage of independence when you can.

Note that when using those formulas, your $x$'s in the regression takes the role of constants, not random variables. Only functions of $y$'s are random variables (the estimates of the parameters are functions of the $y$'s, however).

Improve this answer
$\endgroup$
13
  • $\begingroup$ Much appreciated. I know that $\hat{y}_h$ = $b_0 + b_1 \hat{x_h}$ but I am quite rusty on how to take the variance of multiple terms like this. If I can do this then I believe after that I am meant to substitute $b_0 = Y - b_1X$ or something similar. If I recall manipulating variance I will have $Var(b_0) + Var(b_1) + 2Cov(b1,b0)$ does that seem right? $\endgroup$
    – user26091
    Commented May 29, 2013 at 1:11
  • $\begingroup$ see the edit to my answer. Don't put hats on $x$; the $x$'s are known. edit: ah, your edit to your comment is not quite right, there should be an $x$ in there $\endgroup$
    – Glen_b
    Commented May 29, 2013 at 1:16
  • $\begingroup$ Right, so it should be something like $Var(b_0) + XVar(b_1) + 2Cov(Xb_1,b_0)$? What's strange about all of this is that in my notes somehow I have Cov(b_0,b_1) being the only remaining term that then yields the result through a series of manipulations. $\endgroup$
    – user26091
    Commented May 29, 2013 at 1:19
  • $\begingroup$ Like that, but not that; you have an error. $\endgroup$
    – Glen_b
    Commented May 29, 2013 at 1:20
  • $\begingroup$ Is there any possible way that we could be left with the term $Cov(b_0,b_1)$ alone? In my notes it seems to go that we then have $\sum cov(Y_i / n,k_i Y_i)$ - $\frac{x(\sigma^2)}{S_{xx}}$ by expanding this, which quickly leads to the desired result. $\endgroup$
    – user26091
    Commented May 29, 2013 at 1:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.