# The variance of linear regression estimator $\beta_1$

Can we say

$$\text{Var}(\beta_1) = \text{Var}\left(\frac{\sum (x_i-\bar x)y_i}{\sum (x_i- \bar x)^2}\right) = \left(\frac{\sum (x_i-\bar x)}{\sum (x_i- \bar x)^2}\right)^2 \text{Var}(y_i) \;\;??$$

I am not sure if I can separate the $x$'s i from $\sum (x_i-\bar x)y_i$. This expression seems like a linear combination of $y_i$'s. Is this legit because every $y_i$ follows the same distribution?

• I've used LATEX to give your question mathematical formatting. Please double-check if it stills says what you intended. – Sycorax Nov 2 '14 at 19:18
• Thanks I was editing at the same time. Did I overwrite yours? – user2502240 Nov 2 '14 at 19:19
• It appears that you have overwritten me. This is fine. The most important part is two fold (1) is the question legible to people who might be able to answer and (2) does the formatted text say what you want? – Sycorax Nov 2 '14 at 19:21
• Don't conflate the parameter with its estimate; you'll get very confused. – Glen_b Nov 2 '14 at 21:46
• Briefly: not quite. It's not quite an identical Q but answers here may be of some assistance. – Glen_b Nov 2 '14 at 21:53

This appears to be simple linear regression. If the $x_i$'s are treated as deterministic, then things like "variance" are not associated with them, and so the expression holds, under the additional assumption that the the error term (and hence $y$ also) has identical distribution for all $i$, and also, that the error terms (and hence $y$ also) are independent for all $j\neq i$.

For compactness, denote $$z_i = \frac{x_i-\bar x}{\sum (x_i- \bar x)^2}$$

Then

$$\text{Var}(\beta_1) = \text{Var}\left(\sum z_iy_i\right)$$

The assumption of deterministic $x$'s permits us to treat them as constants. The assumption of independence permits us to set the covariances between $y_i$ and $y_j$ equal to zero. These two give

$$\text{Var}(\beta_1) = \sum z_i^2\text{Var}(y_i)$$

Finally, the assumption of identically distributed $y$'s implies that $\text{Var}(y_i)= \text{Var}(y_j) \;\; \forall i,j$ and so permits us to write

$$\text{Var}(\beta_1) = \text{Var}(y_i)\sum z_i^2$$

There are a few way to approach this. Alecos' method is simple and clear. Here's another:

$\hat{\beta}_1=\frac{\sum (x_i-\bar x)y_i}{\sum (x_i- \bar x)^2}=\frac{\sum (x_i-\bar x)y_i}{S_{xx}}=\frac{\sum d_iy_i}{S_{xx}}$

$\text{Var}(\hat{\beta}_1) = \text{Var}\left(\frac{\sum (x_i-\bar x)y_i}{\sum (x_i- \bar x)^2}\right)=\frac{1}{S_{xx}^2}\text{Var}\left(\sum d_iy_i\right)=\frac{1}{S_{xx}^2}(\sum d_i^2)\text{Var}\left(y_i\right)$

You can then inspect $\sum d_i^2$ in order to do some cancelling out in the formula, and you can substitute a value for the variance of the observations.

Compare with the sample estimate of the standard error, $s_{\hat\beta}$ here, which should be similar up to the two obvious differences.