# How to derive the standard error of linear regression coefficient

For this univariate linear regression model $$y_i = \beta_0 + \beta_1x_i+\epsilon_i$$ given data set $D=\{(x_1,y_1),...,(x_n,y_n)\}$, the coefficient estimates are $$\hat\beta_1=\frac{\sum_ix_iy_i-n\bar x\bar y}{n\bar x^2-\sum_ix_i^2}$$ $$\hat\beta_0=\bar y - \hat\beta_1\bar x$$ Here is my question, according to the book and Wikipedia, the standard error of $\hat\beta_1$ is $$s_{\hat\beta_1}=\sqrt{\frac{\sum_i\hat\epsilon_i^2}{(n-2)\sum_i(x_i-\bar x)^2}}$$ How and why?

• stats.stackexchange.com/questions/44838/… – ocram Feb 9 '14 at 9:14
• @ocram, thanks, but I'm not quite capable of handling matrix stuff, I'll try. – avocado Feb 9 '14 at 9:20
• @ocram, I've already understand how it comes. But still a question: in my post, the standard error has $(n-2)$, where according to your answer, it doesn't, why? – avocado Feb 9 '14 at 9:40

In my post, it is found that $$\widehat{\text{se}}(\hat{b}) = \sqrt{\frac{n \hat{\sigma}^2}{n\sum x_i^2 - (\sum x_i)^2}}.$$ The denominator can be written as $$n \sum_i (x_i - \bar{x})^2$$ Thus, $$\widehat{\text{se}}(\hat{b}) = \sqrt{\frac{\hat{\sigma}^2}{\sum_i (x_i - \bar{x})^2}}$$
With $$\hat{\sigma}^2 = \frac{1}{n-2} \sum_i \hat{\epsilon}_i^2$$ i.e. the Mean Square Error (MSE) in the ANOVA table, we end up with your expression for $\widehat{\text{se}}(\hat{b})$. The $n-2$ term accounts for the loss of 2 degrees of freedom in the estimation of the intercept and the slope.
• I think I get everything else expect the last part. Can you show step by step why $\hat{\sigma}^2 = \frac{1}{n-2} \sum_i \hat{\epsilon}_i^2$ ? I too know it is related to the degrees of freedom, but I do not get the math. – Mappi May 27 '16 at 15:46