# Expected Value and Variance of Estimation of Slope Parameter $\beta_1$ in Simple Linear Regression

I am reading a text, "Probability and Statistics" by Devore. I am looking at 2 items on page 740: the expected value and variance of the estimation of $\beta_1$, which is the slope parameter in the linear regression $Y_i = \beta_0 + \beta_1 X_i + \epsilon_i$. $\epsilon_i$ is a Gaussian($\mu = 0, variance=\sigma^2$) random variable and the $\epsilon_i$ are independent.

The estimate of $\beta_1$ can be expressed as: $\hat{\beta_1} = \frac{\sum (x_i - \bar{x}) (Y_i - \bar{Y})}{\sum(x_i-\bar{x})^2} = \frac{\sum (x_i - \bar{x})Y_i}{S_{xx}}$, where $S_{xx} = \sum (x_i - \bar{x})^2$. So, my question is: how do I derive $E(\hat{\beta_1})$ and $Var(\hat{\beta_1})$? The book has already given the results: $E(\hat{\beta_1}) = \beta_1$ and $Var(\hat{\beta_1}) = \frac{\sigma^2}{S_xx}$.

My work in the derivation: $E\left(\frac{\sum (x_i - \bar{x})Y_i}{S_{xx}}\right) = E\left(\frac{\sum (x_i - \bar{x})(\beta_0 + \beta_1 x_i + \epsilon)}{S_{xx}}\right) = E\left(\frac{\sum (x_i - \bar{x})\beta_1 x_i}{S_{xx}}\right)$, since $\sum(x_i - \bar{x})c = 0$ and $E(c\epsilon) = 0$. But I am stuck.

Also, $Var\left(\frac{\sum (x_i - \bar{x})Y_i}{S_{xx}}\right) = Var\left(\frac{\sum (x_i - \bar{x})(\beta_0 + \beta_1 x_i + \epsilon)}{S_{xx}}\right) = Var\left(\frac{\sum (x_i - \bar{x})\epsilon}{S_{xx}}\right) = Var\left(\frac{\sum (x_i - \bar{x})}{S_{xx}}\right) \sigma^2$. But I am stuck.

Any help would be appreciated. Thanks.

-
My comment on June 22, 2011 in user whuber's answer should include the subscript $i$ in the $\epsilon$'s, and should make use of the fact that the error terms $\epsilon_i$ are independent. –  jrand Mar 20 at 17:57
$\mathrm{Var}(\hat{\beta_1}) = \mathrm{Var}\left(\sum\frac{(x_i - \bar{x})y_i}{S_{xx}}\right) = \mathrm{Var}\left(\sum{\frac{(x_i-\bar{x})\epsilon_i}{S_{xx}}}\right) = \mathrm{Var}\left(\frac{(x_1 - \bar{x})\epsilon_1}{S_{xx}} + \frac{(x_2 - \bar{x})\epsilon_2}{S_{xx}} + \ldots + \frac{(x_n - \bar{x})\epsilon_n}{S_{xx}}\right) = \frac{(x_1 - \bar{x})^2 \sigma^2}{(S_{xx})^2} + \frac{(x_2 - \bar{x})^2 \sigma^2}{(S_{xx})^2} + \ldots + \frac{(x_n - \bar{x})^2 \sigma^2}{(S_{xx})^2} = \sigma^2 \left[\sum{\frac{(x_i-\bar{x})^2}{(S_{xx})^2}}\right] = \frac{\sigma^2}{S_{xx}}$ –  jrand Mar 20 at 17:57
1. $E\left(\frac{\sum (x_i - \bar{x})\beta_1 x_i}{S_{xx}}\right)$ = $\frac{\sum (x_i - \bar{x})\beta_1 x_i}{S_{xx}}$ because everything is constant. The rest is just algebra. Evidently you need to show $\sum (x_i - \bar{x}) x_i = S_{xx}$. Looking at the definition of $S_{xx}$ and comparing the two sides leads one to suspect $\sum(x_i - \bar{x}) \bar{x} = 0$. This follows easily from the definition of $\bar{x}$.
2. $Var\left(\frac{\sum (x_i - \bar{x})\epsilon}{S_{xx}}\right)$ = $\sum \left[\frac{(x_i - \bar{x})^2}{S_{xx}^2}\sigma^2\right]$. It simplifies, using the definition of $S_{xx}$, to the desired result.
For the 2nd point, the variance, the equation should be: $Var\left(\frac{\sum (x_i - \bar{x})\epsilon}{S_{xx}}\right) = Var\left( \frac{(x_1 - \bar{x})\epsilon + (x_2 - \bar{x})\epsilon + \ldots + (x_n - \bar{x})\epsilon}{S_{xx}} \right) = \left( \frac{ \sum_i^{n} (x_i - \bar{x})^2} {S_{xx}^2} \right) \times \sigma^2 = \frac{S_{xx}}{S_{xx}^2} \times \sigma^2 = \frac{\sigma^2}{S_{xx}}$ –  jrand Jun 22 '11 at 16:09
@jrand Yes, that's exactly what I wrote (although your first equality doesn't accomplish anything: it's just a more laborious way to write the summation). The whole point--and the thing worth remembering--is that when $\varepsilon$ is a random variable with a variance and $\lambda$ is constant, $Var(\lambda \varepsilon)$ = $\lambda^2 Var(\varepsilon)$. –  whuber Jun 22 '11 at 16:14
Unless I am getting the notation wrong, this is a false statement: $\sum_i^n (x_i)^2 = \left( \sum_i^n x_i \right)^2$. The quantity on the left hand side is the sum of squares, and the other is the square of the sum. –  jrand Jun 22 '11 at 17:34