# Expected Value and Variance of Estimation of Slope Parameter $\beta_1$ in Simple Linear Regression

I am reading a text, "Probability and Statistics" by Devore. I am looking at 2 items on page 740: the expected value and variance of the estimation of $$\beta_1$$, which is the slope parameter in the linear regression $$Y_i = \beta_0 + \beta_1 X_i + \epsilon_i$$. $$\epsilon_i$$ is a Gaussian($$\mu = 0, variance=\sigma^2$$) random variable and the $$\epsilon_i$$ are independent.

The estimate of $$\beta_1$$ can be expressed as: $$\hat{\beta_1} = \frac{\sum (x_i - \bar{x}) (Y_i - \bar{Y})}{\sum(x_i-\bar{x})^2} = \frac{\sum (x_i - \bar{x})Y_i}{S_{xx}}$$, where $$S_{xx} = \sum (x_i - \bar{x})^2$$. So, my question is: how do I derive $$E(\hat{\beta_1})$$ and $$Var(\hat{\beta_1})$$? The book has already given the results: $$E(\hat{\beta_1}) = \beta_1$$ and $$Var(\hat{\beta_1}) = \frac{\sigma^2}{S_xx}$$.

My work in the derivation: $$E\left(\frac{\sum (x_i - \bar{x})Y_i}{S_{xx}}\right) = E\left(\frac{\sum (x_i - \bar{x})(\beta_0 + \beta_1 x_i + \epsilon)}{S_{xx}}\right) = E\left(\frac{\sum (x_i - \bar{x})\beta_1 x_i}{S_{xx}}\right)$$, since $$\sum(x_i - \bar{x})c = 0$$ and $$E(c\epsilon) = 0$$. But I am stuck.

Also, $$Var\left(\frac{\sum (x_i - \bar{x})Y_i}{S_{xx}}\right) = Var\left(\frac{\sum (x_i - \bar{x})(\beta_0 + \beta_1 x_i + \epsilon)}{S_{xx}}\right) = Var\left(\frac{\sum (x_i - \bar{x})\epsilon}{S_{xx}}\right) = Var\left(\frac{\sum (x_i - \bar{x})}{S_{xx}}\right) \sigma^2$$, but I am stuck.

• My comment on June 22, 2011 in user whuber's answer should include the subscript $i$ in the $\epsilon$'s, and should make use of the fact that the error terms $\epsilon_i$ are independent. Mar 20, 2013 at 17:57
• $\mathrm{Var}(\hat{\beta_1}) = \mathrm{Var}\left(\sum\frac{(x_i - \bar{x})y_i}{S_{xx}}\right) = \mathrm{Var}\left(\sum{\frac{(x_i-\bar{x})\epsilon_i}{S_{xx}}}\right) = \mathrm{Var}\left(\frac{(x_1 - \bar{x})\epsilon_1}{S_{xx}} + \frac{(x_2 - \bar{x})\epsilon_2}{S_{xx}} + \ldots + \frac{(x_n - \bar{x})\epsilon_n}{S_{xx}}\right) = \frac{(x_1 - \bar{x})^2 \sigma^2}{(S_{xx})^2} + \frac{(x_2 - \bar{x})^2 \sigma^2}{(S_{xx})^2} + \ldots + \frac{(x_n - \bar{x})^2 \sigma^2}{(S_{xx})^2} = \sigma^2 \left[\sum{\frac{(x_i-\bar{x})^2}{(S_{xx})^2}}\right] = \frac{\sigma^2}{S_{xx}}$ Mar 20, 2013 at 17:57
• The standard "answer" is an underestimate, it ignores the variance of S_{xx}. Oct 7, 2016 at 4:37
• In the situation being asked about, $X$ is being conditioned on, so it's treated as fixed rather than random Oct 7, 2016 at 7:05

1. $E\left(\frac{\sum (x_i - \bar{x})\beta_1 x_i}{S_{xx}}\right)$ = $\frac{\sum (x_i - \bar{x})\beta_1 x_i}{S_{xx}}$ because everything is constant. The rest is just algebra. Evidently you need to show $\sum (x_i - \bar{x}) x_i = S_{xx}$. Looking at the definition of $S_{xx}$ and comparing the two sides leads one to suspect $\sum(x_i - \bar{x}) \bar{x} = 0$. This follows easily from the definition of $\bar{x}$.
2. $Var\left(\frac{\sum (x_i - \bar{x})\epsilon}{S_{xx}}\right)$ = $\sum \left[\frac{(x_i - \bar{x})^2}{S_{xx}^2}\sigma^2\right]$. It simplifies, using the definition of $S_{xx}$, to the desired result.
• For the 2nd point, the variance, the equation should be: $Var\left(\frac{\sum (x_i - \bar{x})\epsilon}{S_{xx}}\right) = Var\left( \frac{(x_1 - \bar{x})\epsilon + (x_2 - \bar{x})\epsilon + \ldots + (x_n - \bar{x})\epsilon}{S_{xx}} \right) = \left( \frac{ \sum_i^{n} (x_i - \bar{x})^2} {S_{xx}^2} \right) \times \sigma^2 = \frac{S_{xx}}{S_{xx}^2} \times \sigma^2 = \frac{\sigma^2}{S_{xx}}$ Jun 22, 2011 at 16:09
• @jrand Yes, that's exactly what I wrote (although your first equality doesn't accomplish anything: it's just a more laborious way to write the summation). The whole point--and the thing worth remembering--is that when $\varepsilon$ is a random variable with a variance and $\lambda$ is constant, $Var(\lambda \varepsilon)$ = $\lambda^2 Var(\varepsilon)$.
• Unless I am getting the notation wrong, this is a false statement: $\sum_i^n (x_i)^2 = \left( \sum_i^n x_i \right)^2$. The quantity on the left hand side is the sum of squares, and the other is the square of the sum. Jun 22, 2011 at 17:34