I understand degrees of freedom as the number of things that can independently change. And typically, in coming up with the degrees of freedom, if you have n terms, then you just subtract out the number of things which aren't independent (the typical example being the n-1 degrees of freedom in the sample variance). My problem is that I'm not quite seeing how the SSR degrees of freedom (simple linear regression) is just "1" using the previous way of thinking about it. After a little bit of algebra, I can transform it as:

$SSR=\sum_{i=1}^{n}\left ( \hat{y}_i-\bar{y} \right )^2=\sum_{i=1}^{n}\left[\hat{\beta}_0+\hat{\beta}_1x_i- \left(\hat{\beta}_0+\hat{\beta}_1\bar{x} \right) \right]^2=\hat{\beta}^2_1\sum_{i=1}^{n}\left(x_i-\bar{x} \right )^2$

And, I do see that my $\hat{\beta}_1$ is the only estimator (hence one degree of freedom?), however, I'd like to be able to understand this in a similar context of how I understand the n-1 degrees of freedom of the sample variance, namely "adding up n terms, subtracting out the number of terms that aren't independent" to somehow get 1.

  • $\begingroup$ Where do you find an assertion that SSR has just one df? What exactly does it say? $\endgroup$ – whuber Sep 19 '14 at 14:39
  • $\begingroup$ This doesn't answer you question, but as far as I know the division by (n-1) rather than by (n) in the calculation of sample variance is to make the estimator unbiased; it doesn't have to do with degrees of freedom. $\endgroup$ – nrussell Sep 19 '14 at 14:40
  • $\begingroup$ @nrussell You're correct in that it makes the estimator unbiased. However, the DF are n-1 because we're estimating the true mean using the sample mean, hence one degree of freedom is lost. $\endgroup$ – Eric Sep 19 '14 at 14:44
  • $\begingroup$ @whuber It's from the ANOVA relationship SST = SSR + SSE: the sum of squares of the residuals and the sum of squares of the error add up to the total sum of squares (which I believe by algebra). In addition, the DF of the SSR + the DF of the SSE equal the total DF; the DF of the SST are n, the DF of the SSE are n-1. $\endgroup$ – Eric Sep 19 '14 at 14:46
  • $\begingroup$ Thanks, Eric. I had taken "SSR" in the sense in which you mean "SSE" and did not read the formula carefully. The terms being squared and summed in your SSR formula are not the residuals: they are the differences between the predicted values (which are based on two estimated parameters) and the grand mean of the response (based on one estimated parameter). This mischaracterization might be at the root of your question. $\endgroup$ – whuber Sep 19 '14 at 14:58

I just think of it in terms of the fitted line having 2 parameters, the intercept and the slope. But the intercept doesn't count after we adjust for the mean (which we do in obtaining SStotal), so that leaves just one df for regression.


Not the answer you're looking for? Browse other questions tagged or ask your own question.