# How to estimate standard error for parameters in a linear regression

> summary(model)

Call:
lm(formula = y ~ x, data = data)

Residuals:
1          2          3          4          5          6          7
-7.143e-01  1.429e+00 -4.286e-01 -2.857e-01 -1.429e-01 -4.996e-16  1.429e-01

Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept)   3.1429     0.9035   3.479  0.01769 *
x             0.8571     0.1429   6.000  0.00185 **
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 0.7559 on 5 degrees of freedom
Multiple R-squared:  0.878, Adjusted R-squared:  0.8537
F-statistic:    36 on 1 and 5 DF,  p-value: 0.001846


What is the value 0.9035 and 0.1429? How to compute these values?

• The current answers tell you what's going on in the software. Is your interest in how to derive the equations giving those values, however (theory)?
– Dave
Commented Oct 22, 2019 at 13:48
• Yes, also I want the intuition behind it Commented Oct 29, 2019 at 6:01

What is the value 0.9035 and 0.1429?

With the OLS estimation of theoretical parameters $$\beta_0$$ and $$\beta_1$$ of linear regression $$y = \beta_0 + \beta_1 x + \varepsilon$$ based on a sample the expected value of the error of your estimation is $$0.9035$$ and $$0.1429$$.

With other words: you have missed the 'real' parameters on average by this value.

How to compute these values?

$$\hat{\text{s.e.}}_{\hat{\beta}_j} = \sqrt{s^2 \left( X^T X \right)^{-1}_{jj}}$$

where

$$s^2 = \frac{\hat{\varepsilon}^T \hat{\varepsilon}}{n - p} \qquad\qquad (\hat{\varepsilon} = y - \hat{y})$$

The above is the general case with $$p$$ independent variable ($$x_1, x_1, \ldots x_p$$). For simple linear regression (with just $$1$$ independent variable ($$\hat{y} = \hat{\beta}_0 + \hat{\beta}_1 x$$)):

$$\hat{\text{s.e.}}_{\hat{\beta}_1} = \sqrt{\frac{\frac{1}{n - 2}\sum_{i = 1}^n \hat{\varepsilon}_i^2}{\sum_{i = 1}^n \left( x_i - \overline{x} \right)^2}} \qquad\qquad (\hat{\varepsilon}_i = y_i - \hat{y}_i)$$

$$\hat{\text{s.e.}}_{\hat{\beta}_0} = \hat{\text{s.e.}}_{\hat{\beta}_1} \sqrt{\frac{1}{n} \sum_{i = 1}^n x_i}$$

The primary result of your regression is the formula

$$y = .85\times x + 3.14 + \epsilon$$

However, this does not say how precise your estimation of the values $$.85$$ and $$3.14$$ is. If you had hundreds of data points very close on a straight line, you'd be very confident, that theses numbers are pretty much correct. If you derived these numbers form only 5 data points that are all over the place, you'd doubt the correctness of these estimates.

That is why R gives you standard errors of the coefficients as well as $$p$$-values for each coefficient. The best way to compute those is the lm-function that you have used to compute them.