2
$\begingroup$

Why is the degrees of freedom for Simple Linear Regression $n-2$?

Specifically looking at a hypothesis $t$-test to determine if there's a relationship between the one independent variable, $x$, and the dependent variable $y$.

I would assume it would be $n-k$ where $k = 1$ in the case where we're using the slope in the simple linear regression equation as the parameter.

$\endgroup$
2
  • 2
    $\begingroup$ Don't forget the intercept. $\endgroup$
    – whuber
    Oct 31, 2017 at 22:02
  • $\begingroup$ Because you have two parameters. $\endgroup$
    – SmallChess
    Oct 31, 2017 at 22:03

2 Answers 2

4
$\begingroup$

You are correct that the degrees of freedom are $n-k$, however, in simple linear regression you estimate both a y-intercept and a slope, so $k=2$. Even though we generally don't worry about testing the intercept, it still uses up a degree of freedom, the slope would be very different and have a very different interpretation if we did not estimate an intercept along with the slope.

$\endgroup$
3
$\begingroup$

If our model is $y_{i} = \beta_{0} + \beta_{x}x_{i} + \varepsilon_{i}$, and we want to test, for example, $H_{0}: \beta_{x} = 0$, against the alternative $\beta_{x} \ne 0$... we require an estimate: $\hat{\beta}$.

$$\hat{\beta} = \frac{\sum_{i=1}^{n}{\left(x_{i} - \bar{x}\right)\times \left(y_{i} - \bar{y}\right)}}{\sum_{i=1}^{n}{\left(x_{i} - \bar{x}\right)^{2}}}$$

Notice that to obtain $\hat{\beta}$ we require an estimate of two quantities: $\mu_{x}$, and $\mu_{y}$, which we have in $\bar{x}$ and $\bar{y}$. We lose a degree of freedom for each of these estimates.

$\endgroup$
2
  • 1
    $\begingroup$ I suspect the root of the question is the thought that if we're testing just one parameter, and the other is of no interest, then why should the presence of that other parameter play any role in the test? Although pointing out that the other parameter is estimated anyway is a useful heuristic for this particular model, it's ultimately not very convincing, IMHO. Greg Snow ends his answer with an excellent observation about this. $\endgroup$
    – whuber
    Oct 31, 2017 at 22:24
  • 1
    $\begingroup$ @whuber I appreciate this, and Greg Snow's answer also... I think the "not very convincing" in my answer is from its coming "long way around" to the intercept which also depends on the estimates of $\bar{x}$ and $\bar{y}$. $\endgroup$
    – Alexis
    Oct 31, 2017 at 22:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.