# Proof: Linear Regression

I am at a loss how might one prove $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$. Note that $\hat{y}_h$ = $b_0 + b_1X_h$ which is a regression line estimate at some given $X_h$.

Moreover, what is the second term actually saying? I understand the first is just the variance divided by the degrees of freedom but the second is more complex it seems.

• please add the self-study tag – Glen_b May 29 '13 at 0:52

Here's an outline:

• Write down the formula for $\hat{y}_h$.

• Take the variance of it.

The second involves rather similar steps to your earlier variance question.

The second term is related to the effect of the uncertainty in the slope estimate on the uncertainty in the fitted value; this gets bigger the further you get from the mean $x$.

I am quite rusty on how to take the variance of multiple terms like this

Just keep using basic properties of things you know about, like:

$\text{Var}(X+Y) = \text{Var}(X) + \text{Var}(Y) + 2\, \text{Cov}(X,Y)$

$\text{Var}(aX) = a^2\text{Var}(X)$

$\text{Cov}(aX,bY)= ab\,\text{Cov}(X,Y)$

and so on

Take advantage of independence when you can.

Note that when using those formulas, your $x$'s in the regression takes the role of constants, not random variables. Only functions of $y$'s are random variables (the estimates of the parameters are functions of the $y$'s, however).

• Much appreciated. I know that $\hat{y}_h$ = $b_0 + b_1 \hat{x_h}$ but I am quite rusty on how to take the variance of multiple terms like this. If I can do this then I believe after that I am meant to substitute $b_0 = Y - b_1X$ or something similar. If I recall manipulating variance I will have $Var(b_0) + Var(b_1) + 2Cov(b1,b0)$ does that seem right? – user26091 May 29 '13 at 1:11
• see the edit to my answer. Don't put hats on $x$; the $x$'s are known. edit: ah, your edit to your comment is not quite right, there should be an $x$ in there – Glen_b May 29 '13 at 1:16
• Right, so it should be something like $Var(b_0) + XVar(b_1) + 2Cov(Xb_1,b_0)$? What's strange about all of this is that in my notes somehow I have Cov(b_0,b_1) being the only remaining term that then yields the result through a series of manipulations. – user26091 May 29 '13 at 1:19
• Like that, but not that; you have an error. – Glen_b May 29 '13 at 1:20
• Is there any possible way that we could be left with the term $Cov(b_0,b_1)$ alone? In my notes it seems to go that we then have $\sum cov(Y_i / n,k_i Y_i)$ - $\frac{x(\sigma^2)}{S_{xx}}$ by expanding this, which quickly leads to the desired result. – user26091 May 29 '13 at 1:28