# How to derive the standard error of linear regression coefficient

For this univariate linear regression model $$y_i = \beta_0 + \beta_1x_i+\epsilon_i$$ given data set $D=\{(x_1,y_1),...,(x_n,y_n)\}$, the coefficient estimates are $$\hat\beta_1=\frac{\sum_ix_iy_i-n\bar x\bar y}{n\bar x^2-\sum_ix_i^2}$$ $$\hat\beta_0=\bar y - \hat\beta_1\bar x$$ Here is my question, according to the book and Wikipedia, the standard error of $\hat\beta_1$ is $$s_{\hat\beta_1}=\sqrt{\frac{\sum_i\hat\epsilon_i^2}{(n-2)\sum_i(x_i-\bar x)^2}}$$ How and why?

• stats.stackexchange.com/questions/44838/… Feb 9 '14 at 9:14
• @ocram, thanks, but I'm not quite capable of handling matrix stuff, I'll try. Feb 9 '14 at 9:20
• @ocram, I've already understand how it comes. But still a question: in my post, the standard error has $(n-2)$, where according to your answer, it doesn't, why? Feb 9 '14 at 9:40

3rd comment above: I've already understand how it comes. But still a question: in my post, the standard error has (n−2), where according to your answer, it doesn't, why?

In my post, it is found that $$\widehat{\text{se}}(\hat{b}) = \sqrt{\frac{n \hat{\sigma}^2}{n\sum x_i^2 - (\sum x_i)^2}}.$$ The denominator can be written as $$n \sum_i (x_i - \bar{x})^2$$ Thus, $$\widehat{\text{se}}(\hat{b}) = \sqrt{\frac{\hat{\sigma}^2}{\sum_i (x_i - \bar{x})^2}}$$

With $$\hat{\sigma}^2 = \frac{1}{n-2} \sum_i \hat{\epsilon}_i^2$$ i.e. the Mean Square Error (MSE) in the ANOVA table, we end up with your expression for $\widehat{\text{se}}(\hat{b})$. The $n-2$ term accounts for the loss of 2 degrees of freedom in the estimation of the intercept and the slope.

• I think I get everything else expect the last part. Can you show step by step why $\hat{\sigma}^2 = \frac{1}{n-2} \sum_i \hat{\epsilon}_i^2$ ? I too know it is related to the degrees of freedom, but I do not get the math. May 27 '16 at 15:46

another way of thinking about the n-2 df is that it's because we use 2 means to estimate the slope coefficient (the mean of Y and X)

df from Wikipedia: "...In general, the degrees of freedom of an estimate of a parameter are equal to the number of independent scores that go into the estimate minus the number of parameters used as intermediate steps in the estimation of the parameter itself ."

• This isn't really a derivation as such, though it is an intuition. For some subtleties related to this, though, see How to understand degrees of freedom? Mar 23 '17 at 12:04