Every statistic (a function of data, e.g., $\bar{X}$) has its own degree of freedom (DF). My question is: why often times, you want to estimate some parameter $\theta$ and you come up with some estimator $\hat{\theta}$, which has the form of sum of squares, like $\sum_{i=1}^{n}(x_i-\bar{x})^2$, then you divide it by its correct DF, then you obtained a "good" estimator of $\theta$? I know mathematically, one can prove that how "good" this estimator (after divided by its DF) is. But my question is exactly, why it is so amazing that the DF seems to make the "just right adjustment" to obtain a "good" estimator? For example, the statistic $\sum_{i=1}^{n}(x_i-\bar{x})^2$, then you divide it by $n-1$, you obtain an unbiased estimator for the population variance, but this statistic $\frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})^2$ also have DF $n-1$, why not you further divide it by $n-1$? I know mathematically it is a silly question, but I just couldn't understand DF in a natural way.

Another example is, when we compute the confidence interval (CI) of sample mean, we use the formula $\bar{X}\pm t_{\alpha/2}(n-1)\frac{s}{\sqrt{n}}$, where $s=\frac{1}{n-1}\sum_{i=1}^{n}(x_i-\bar{x})^2$. Note that in this expression, the DF $n-1$ appears in 2 places, once is in the MSE, $s$, and another is in the $t$ distribution. We see that if DF gets smaller, then MSE will increase so the CI will be wider, then the CI is less precise. In addition, when DF gets smaller, the $t$ distribution becomes heavy tail, which also widens the CI. So it seems that DF is very important in determining the precision of an estimator.

I can only see the points of DF at this level due to my shallow understanding. But I strongly believe there should be a more natural way to view this concept.



Browse other questions tagged or ask your own question.