Proving $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$

I have asked in another question how $\text{Var}{(\hat{y}_h)} = \sigma^2 \left(\frac{1}{n} + \frac{(x_h-\bar{x})^2}{S_{xx}}\right)$. Note that $\hat{y}_h$ = $b_0 + b_1X_h$ which is a regression line estimate at some given $X_h$.

This question concerns why the term $Cov(b_0,b_1)$ alone yields the RHS. Substituting $b_0 = Y - b_1X$ we get that $Cov(Y,b_1) - XCov(b_1,b_1)$ = $Cov(\frac{\sum{Y_i}}{n},\sum k_iY_i) - XVar{(b_1)}$. Here X and Y without subscript are arithmetic means.

We can then rearrange to obtain $\sum \frac{k_i Var(Y_i)}{n} - \frac{X\sigma^2}{S_{xx}}$ which quickly yields the desired result. My question is, why does this work? This single term does not seem like it should alone yield the RHS. Have I made an error in algebra?

• Could anyone explain why this was downvoted, it seems like a reasonable question to me. – user26091 May 29 '13 at 14:33
• I would start by searching our site: this formula has been quoted extensively and therefore likely has several different demonstrations. – whuber Nov 1 '18 at 19:17
• Also related: stats.stackexchange.com/questions/115011/…. – StubbornAtom Nov 1 '18 at 19:20

$(1)\ E(\hat{Y_h}) = E(b_0 + b_1X_h) = \beta_0 +\beta_1X_h$

$(2)\ var(\hat{Y_h}) = var(b_0 + b_1X_h)$

An alternate (but equivalent) version of the regression model can be written as:

$Y_i = \beta_0X_0 + \beta_1X_1 + \epsilon_i$

This model associates an X variable with each coefficient $(where X_0 = 1)$

Al alternate modification is to use the deviation $X_i -\bar{X}$ rather than $X_i$

So the model can be written as:

$Y_i = \beta_0^* + \beta_1(X_i - \bar{X}) + \epsilon_i$

where $(3)\ \beta_0^* = \beta_0 + \beta_1\bar{X}$

These models can be used interchangably.

We know from the normal equations:

$\Sigma Y_i = nb_0 + b_1\Sigma X_i$

solving for $b_0$

$(4)\ b_0 = \bar{Y} - b_1\bar{X}$

So substituting from (3) and (4):

$b_0^* = b_0 + b_1\bar{X} = (\bar{Y} - b_1\bar{X}) + b_1\bar{X} = \bar{Y}$

$(5)\ var(\hat{Y_h}) = var(b_0 + b_1X_h) = var(\bar{Y} + b_1(X_h - \bar{X}))$

using:

$var(\bar{Y}) = \frac{\sigma^2}{n}$

$var(aX) = a^2var(X)$

and

$var(X + Y) = Var(X) + Var(Y) + 2Cov(X,Y)$

So:

= $var(\bar{Y}) +(X_h - \bar{X})^2var(b_1) + 2(X_h-\bar{X})cov(\bar{Y},b_1)$

we use the fact that $Cov(\bar{Y},b_1) = 0$ due to the i.i.d assumption on $\epsilon_i$

$= \frac{\sigma^2}{n} + (X_h-\bar{X})^2\frac{\sigma^2}{\Sigma(X_i-\bar{X})^2}$

$= \sigma^2[\frac{1}{n} + \frac{(X_h - \bar{X})^2}{\Sigma(X_i - \bar{X})^2}]$

• All of this can be found in Chapter 1 and 2 of Applied Linear Regression Models by Kutner, Nachtsheim and Neter. That's where I was sifting through to find stuff. – Eric Peterson May 29 '13 at 12:49
• including several pages on the normal equations which shows how $b_1 = \frac{\Sigma(X_i - \bar{X})(Y_i - \bar{Y})}{\Sigma(X_i - \bar{X})^2}$ to get you the last two steps of my proof. – Eric Peterson May 29 '13 at 13:07
• Thank you Clark, very clear. I will see if I can find that book. Enjoy your trip to Wally World this summer. – user26091 May 29 '13 at 15:34

Provided that

$$\text{Var}(\hat{y}_{x_0}) = \displaystyle\frac{\sigma^2\sum x_i^2}{n\sum(x_i - \bar{x})^2} + \displaystyle\frac{\sigma^2x_0^2}{\sum(x_i - \bar{x})^2} - \displaystyle\frac{2x_0\sigma^2\bar{x}}{\sum(x_i - \bar{x})^2}$$

The term, $$\displaystyle\frac{\sigma^2\sum x_i^2}{n\sum(x_i - \bar{x})^2}$$ is the most troublesome and can be broken down using basic arithmetic and knowing that playing with $$\sum x_i^2$$ is dangerous. So,

$$\displaystyle\frac{\sigma^2(\sum x_i^2 + 2n\bar{x}^2 - 2n\bar{x}^2)}{n\sum(x_i - \bar{x})^2} \rightarrow \displaystyle\frac{\sigma^2(n\bar{x}^2 + \sum x_i^2 + \bar{x}^2\sum(1) - 2n\bar{x}\{\sum(x_i)/n\})}{n\sum(x_i - \bar{x})^2}$$ Where $$n = \sum(1)$$ and $$\bar{x} = \sum(x_i)/n$$ which is basic. $$\displaystyle\frac{\sigma^2(n\bar{x}^2 + \sum(x_i^2 + \bar{x}^2 - 2\bar{x}x_i))}{n\sum(x_i - \bar{x})^2} \rightarrow \displaystyle\frac{\sigma^2(n\bar{x}^2 + \sum(x_i - \bar{x})^2)}{n\sum(x_i - \bar{x})^2}$$ Here $$\sum(x_i - \bar{x})^2 = S_{xx}$$ so, our final term is $$\displaystyle\frac{\sigma^2(n\bar{x}^2 + S_{xx})}{nS_{xx}} \rightarrow \displaystyle\frac{\sigma^2\bar{x}^2}{S_{xx}} + \displaystyle\frac{\sigma^2}{n}$$ So, our original equation becomes $$\text{Var}(\hat{y}_{x_0}) = \displaystyle\frac{\sigma^2\bar{x}^2}{S_{xx}} + \displaystyle\frac{\sigma^2}{n} + \displaystyle\frac{\sigma^2x_0^2}{S_{xx}} - \displaystyle\frac{2x_0\sigma^2\bar{x}}{S_{xx}}$$ $$\text{Var}(\hat{y}_{x_0}) = \sigma^2\left[\displaystyle\frac{1}{n} + \displaystyle\frac{\bar{x}^2 + x_0^2-2x_0\bar{x}}{S_{xx}}\right] \rightarrow \text{Var}(\hat{y}_{x_0}) = \sigma^2\left[\displaystyle\frac{1}{n} + \displaystyle\frac{(x_0 - \bar{x})^2}{S_{xx}}\right]$$

• Hello and thanks for your input ! I think that this $$\text{Var}(\hat{y}_{x_0}) = \displaystyle\frac{\sigma^2\sum x_i^2}{n\sum(x_i - \bar{x})^2} + \displaystyle\frac{\sigma^2x_0^2}{\sum(x_i - \bar{x})^2} - \displaystyle\frac{2x_0\sigma^2\bar{x}}{\sum(x_i - \bar{x})^2}$$ needs to be proven as well though. – Rebellos Nov 2 '18 at 8:24
• My bad, I'll see to it ASAP. – Syed Ali Nov 2 '18 at 10:41
• Did you check on how to prove the big expression? – Rebellos Nov 3 '18 at 15:18
• No luck with that one – Syed Ali Nov 4 '18 at 12:00