1
$\begingroup$

Why is the degrees of freedom for Simple Linear Regression $n-2$?

Specifically looking at a hypothesis $t$-test to determine if there's a relationship between the one independent variable, $x$, and the dependent variable $y$.

I would assume it would be $n-k$ where $k = 1$ in the case where we're using the slope in the simple linear regression equation as the parameter.

$\endgroup$
  • 2
    $\begingroup$ Don't forget the intercept. $\endgroup$ – whuber Oct 31 '17 at 22:02
  • $\begingroup$ Because you have two parameters. $\endgroup$ – SmallChess Oct 31 '17 at 22:03
3
$\begingroup$

You are correct that the degrees of freedom are $n-k$, however, in simple linear regression you estimate both a y-intercept and a slope, so $k=2$. Even though we generally don't worry about testing the intercept, it still uses up a degree of freedom, the slope would be very different and have a very different interpretation if we did not estimate an intercept along with the slope.

$\endgroup$
2
$\begingroup$

If our model is $y_{i} = \beta_{0} + \beta_{x}x_{i} + \varepsilon_{i}$, and we want to test, for example, $H_{0}: \beta_{x} = 0$, against the alternative $\beta_{x} \ne 0$... we require an estimate: $\hat{\beta}$.

$$\hat{\beta} = \frac{\sum_{i=1}^{n}{\left(x_{i} - \bar{x}\right)\times \left(y_{i} - \bar{y}\right)}}{\sum_{i=1}^{n}{\left(x_{i} - \bar{x}\right)^{2}}}$$

Notice that to obtain $\hat{\beta}$ we require an estimate of two quantities: $\mu_{x}$, and $\mu_{y}$, which we have in $\bar{x}$ and $\bar{y}$. We lose a degree of freedom for each of these estimates.

$\endgroup$
  • 1
    $\begingroup$ I suspect the root of the question is the thought that if we're testing just one parameter, and the other is of no interest, then why should the presence of that other parameter play any role in the test? Although pointing out that the other parameter is estimated anyway is a useful heuristic for this particular model, it's ultimately not very convincing, IMHO. Greg Snow ends his answer with an excellent observation about this. $\endgroup$ – whuber Oct 31 '17 at 22:24
  • $\begingroup$ @whuber I appreciate this, and Greg Snow's answer also... I think the "not very convincing" in my answer is from its coming "long way around" to the intercept which also depends on the estimates of $\bar{x}$ and $\bar{y}$. $\endgroup$ – Alexis Oct 31 '17 at 22:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.