5
$\begingroup$

On Wikipedia, it says

If $X_{i};i=1,\ldots ,n$ are independent normal $(\mu ,\sigma ^{2})$ random variables, the statistic

$$\frac{\sum\limits^n_{i=1}(X_i-\bar{X})^2}{\sigma^2}$$

follows a chi-squared distribution with $n − 1$ degrees of freedom. Here, the degrees of freedom arises from the residual sum-of-squares in the numerator, and in turn the $n − 1$ degrees of freedom of the underlying residual vector $\{X_{i}-{\bar {X}}\}$.

With that in mind, I'd like to put forward the following example:

Suppose we choose an integer, $10$. Let's say our goal is to choose some $n$ values that sum to $10$. Let's say $n=3$. I freely choose $6$ and then $3$, but now I am forced to choose $1$. Hence I have $2$ degrees of freedom. Or more generally, $n-1$ degrees of freedom.

In this example, it's clear that the final choice is not a free one, because we have some target value, and so once we've made $n-1$ choices for the summands, there is only one possible value that ensures we satisfy the target sum value constraint.

Now, going back to case of the residual sum-of-squares on the numerator. If this were $\sum\limits^n_{i=1}(X_i-\bar{X})$ instead (notice I've dropped the square), then our target is $0$. So again, like in the example above, we have $n-1$ degrees of freedom because we can make free choices for $X_1\ldots X_{n-1}$, but then $X_n$ is forced. However, once we square each $(X_i - \bar{X})$, we lose the target value of $0$.

So how do we know that the degrees of freedom is $n-1$ without a target value? If there is no target value, why are we not able to make $n$ free choices?

$\endgroup$
6
  • $\begingroup$ You have a target value, namely, the sample mean $\bar{X}$. $\endgroup$ – jbowman Jul 6 '19 at 21:35
  • $\begingroup$ This has been answered before. See links in the right margin of this page under 'Related'. In particular, technical details are explored here. I continue this Comment in answer format with a simulation in R: to illustrate that there are $n-1$ degrees of freedom--not to prove the result. $\endgroup$ – BruceET Jul 6 '19 at 22:11
  • $\begingroup$ @jbowman Are you able to elaborate on why $\bar{X}$ is a target value? As I see it, we've already freely chosen $X_1 \ldots X_n$ before we even compute $\bar{X}$. $\endgroup$ – Data Jul 7 '19 at 15:51
  • $\begingroup$ You haven't freely chosen $n$ values of $(x_i-\bar{x})$, because the $x_i$ have to sum to $n\bar{x}$. Just as with your paragraph starting with "Suppose we choose..."... well, in your example, $\bar{x} = 10/3$. The sentence beginning "However, once we square each"... seems to imply you think that you don't have to pay attention to the fact that $\sum x_i/n = \bar{x}$ any more, but you do. $\endgroup$ – jbowman Jul 7 '19 at 17:24
  • $\begingroup$ @jbowman Thank you for that explanation. I would like to know if I'm thinking about it clearly now. Suppose we choose $x_1=1, x_2=2, x_3=3$, so that $\bar{x} = \sum x_i /n = 2.$ In determining $\bar{x}$ we had $n$ degrees of freedom because I can choose any $n$ numbers I wanted. Now with the $(x_i - \bar{x})$, we can choose a new set of $x_i$, so long as we satisfy $\bar{x} = 2$. Hence we have a target. The sum $\sum (x_i - \bar{x})^2$ doesn't have a particular target, but the $x_i$ do. E.g. I can choose $x_1=1,x_2=1$, and to satisfy $\bar{x}=2$, I must choose $x_3=4$. Hence $n-1$ DOF. $\endgroup$ – Data Jul 7 '19 at 18:02
5
$\begingroup$

Consider samples of size $n= 5$ from a standard normal distribution. then $Q =(n-1)S^2 \sim \mathsf{Chisq}(\nu=4),$ not $\mathsf{Chisq}(\nu=5).$

set.seed(706);  m = 10^6;  n = 5
q = replicate( m, (n-1)*var(rnorm(n)) )
mean(q)
[1] 4.002257  # aprx E(Q) = 4
hdr = "Simulated Dist'n of Q fits CHISQ(4)[blue], not CHISQ(5) [red]"
hist(q, prob=T, br=50, col="skyblue2", main=hdr)
  curve(dchisq(x, 4), add=T, lwd=2, col="blue")
  curve(dchisq(x, 5), add=T, lwd=3, col="red", lty="dotted")

enter image description here

Note: If you consider $n = 2,$ then it is easy to verify that $\bar X$ is a function of $X_1 + X_2$ and independently, $S^2$ is a function of $X_1 - X_2$ so that $S^2 \sim \mathsf{Chisq}(1).$ One proof for $n > 2$ shows that $\bar X$ is a function of a vector in one dimention, and that (orthogonally) $S^2$ is a function of vectors in $n-1$ dimensions.

$\endgroup$
1
  • 1
    $\begingroup$ The note does it. $\endgroup$ – Sextus Empiricus Jul 7 '19 at 0:01

Not the answer you're looking for? Browse other questions tagged or ask your own question.