4
$\begingroup$

I went through this thread on degrees of freedom: How to understand degrees of freedom?, and the great answers in it, but then I read the following on Wikipedia on the article about regression:

Statistical assumptions

When the number of measurements, N, is larger than the number of unknown parameters, k, and the measurement errors εi are normally distributed then the excess of information contained in (N − k) measurements is used to make statistical predictions about the unknown parameters. This excess of information is referred to as the degrees of freedom of the regression.

Given this definition, if $N$ increases, the degrees of freedom increase as well, but intuitively that would make the problem more constrained (we have more information per parameter). Why is N-k then called degrees of freedom, and it isn't the other way around e.g. (k-N)?

Improve this question
$\endgroup$
6
$\begingroup$

You may be confused between degrees of freedom attributed to different things.

We would not use negative numbers to count; but there are two sides to the ledger.

In common situations, the data degrees of freedom will be $N$, say.

The model degrees of freedom -- the degrees of freedom the model has to fit the data -- is $k$, and the residual degrees of freedom is what's left over: $N-k$. That $k$ may often be partitioned into various components of the model.

Any of them might be called "the" degrees of freedom depending on what, exactly, is being discussed.

Indeed, we use 'degrees of freedom' more broadly still, whence the appearance of noninteger degrees of freedom for some kinds of models, and references to things like "researcher degrees of freedom".

Improve this answer
$\endgroup$
8
  • 1
    $\begingroup$ I really hope you could write a full-fledged answer for what's DF as every time people ask about what's DF, the mods just redirect to whuber's how to understand DF. It's surely an insightful answer but it is not very intuitive to audience who try to correctly understand what DF is, IMHO. $\endgroup$
    – stucash
    Jul 25 '20 at 5:52
  • $\begingroup$ I'd be very happy for you to post an answer. If you write a very good one (and point me toward it, wherever it gets posted), I'll be seeking to award it a bounty. $\endgroup$
    – Glen_b
    Jul 25 '20 at 6:13
  • $\begingroup$ Thanks for reverting back. And I actually was thinking out loud :) I understand your points above: a good answer is hardly not a combination of all above. After flipping many pages tagged Degree of Freedom on StatEx I guess I was a bit lost. I'll take back the phrase full-fledged and I am sure I can't provide a good enough answer that doesn't fall short of your expectation and is not prone to errors (if not a lot). You are right, after I failed to grasp what whuber's explained in his answer, I started doing my own research. I guess I have become relying too much on StatEx and got lazy :P $\endgroup$
    – stucash
    Jul 25 '20 at 6:29
  • 1
    $\begingroup$ I'm not familiar with StatEx (is it a book, a website, or something else?) -- can you provide a link to the thing you were looking at in relation to degrees of freedom? (I tried a couple of search engines but only got links to information about morphine or exchange rates) $\endgroup$
    – Glen_b
    Jul 25 '20 at 6:39
  • 1
    $\begingroup$ I am actually a few hours into writing some R utilities programmes for volatility analysis... I meant Cross Validated (which is apparently a sub-site of Stack Exchange where I mentally replaced Stack with Stat). I am terribly sorry. $\endgroup$
    – stucash
    Jul 25 '20 at 6:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.