I am dealing right now with a lot of distributions, e.g., $F$, $t$, $\chi^2$.
I was wondering why do these degrees of freedom signify for distributions such as the $F(m,n)$ distribution?
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.Sign up to join this community
Here is a less technical answer, perhaps more accessible to people with modest mathematical preparation.
The term degrees of freedom (df) is used in connection with various test statistics but its meaning varies from one statistical test to the next. Some tests do not have degrees of freedom associated with the test statistic (e.g., Fisher's Exact Test or the z test). When we do a z test, the z value we calculate based on our data can be interpreted based on a single table of critical z values, no matter how large or small our sample(s). Another way to say this is that there is one z distribution. That is not so for some other tests (e.g., F or t or χ2).
The reason many test statistics need to be interpreted in light of df is that the (theoretical) distribution of values of the test statistic, assuming the null hypothesis is true, depends on sample size or number of groups, or both, or some other fact about the data gathered. In doing a t-test, the distribution of t values depends on the sample size, so when we evaluate the t value we calculate from the observed data we need to compare it to t values expected based on the same sample size as our data. Similarly, the distribution of values of F in an Analysis of Variance (assuming the null hypothesis is true) depends on both sample size and the number of groups. So to interpret the F value we calculate from our data we need to use tables of F values that are based on the same sample size and the same number of groups as we have in our data. Saying this differently, F tests (i.e., ANOVAs) and t-tests and χ2 tests each require a family of curves to help us interpret the t or F or χ2 value we calculate based on our data. We choose from among these families of curves based on values (i.e. df's) so that the probabilities we read from the tables are appropriate for our data. (Of course, most computer programs do this for us.)
The F distribution is the ratio of two central chi-square distribution. The m is the degrees of freedom associated with the chi-square random variable that represents the numerator and the n is the degrees of freedom of the chi-square for the denominator. To complete the answer to your question I need to explain the chi-square degrees of freedom. A chi-square distribution with n degrees of freedom can be represented as the sum of squares of n independent N(0,1) random variables. So the degrees of freedom can be looked at as the number of normal random variable that appear in the sum.
Now this will change if these normals include estimated parameters. Suppose for example we have n independent N(m,1) random variables X$_i$ i=1,2,...,n. Then let X$_b$ be the sample mean = ∑X$_i$/n.
Now compute S$^2$ = ∑(X$_i$-X$_b$)$^2$. This S$^2$ will have a chi-square distribution but with n-1 degrees of freedom. In this case we are still summing n, squared N(0,1) random variables. But the difference here is that they are not independent because each one is formed using the same X$_b$. So for the chi-square it is often said that the degrees of freedom equals the number of terms in the sum minus the number of parameters estimated.
In the case of the t distribution we have a N(0,σ$^2$) divided by V where V is the sample estimate of σ. V is proportional to a chi-square with n-1 degrees of freedom where n is the sample size. The degrees of freedom for the t is the degrees of freedom for the chi-square random variable that is involved in the calculation of V.