# Why is $\frac{\sum^n_{i=1}(X_i-\bar{X})^2}{\sigma^2}$chi-square distributed with $n-1$ degrees of freedom?

On Wikipedia, it says

If $$X_{i};i=1,\ldots ,n$$ are independent normal $$(\mu ,\sigma ^{2})$$ random variables, the statistic

$$\frac{\sum\limits^n_{i=1}(X_i-\bar{X})^2}{\sigma^2}$$

follows a chi-squared distribution with $$n − 1$$ degrees of freedom. Here, the degrees of freedom arises from the residual sum-of-squares in the numerator, and in turn the $$n − 1$$ degrees of freedom of the underlying residual vector $$\{X_{i}-{\bar {X}}\}$$.

With that in mind, I'd like to put forward the following example:

Suppose we choose an integer, $$10$$. Let's say our goal is to choose some $$n$$ values that sum to $$10$$. Let's say $$n=3$$. I freely choose $$6$$ and then $$3$$, but now I am forced to choose $$1$$. Hence I have $$2$$ degrees of freedom. Or more generally, $$n-1$$ degrees of freedom.

In this example, it's clear that the final choice is not a free one, because we have some target value, and so once we've made $$n-1$$ choices for the summands, there is only one possible value that ensures we satisfy the target sum value constraint.

Now, going back to case of the residual sum-of-squares on the numerator. If this were $$\sum\limits^n_{i=1}(X_i-\bar{X})$$ instead (notice I've dropped the square), then our target is $$0$$. So again, like in the example above, we have $$n-1$$ degrees of freedom because we can make free choices for $$X_1\ldots X_{n-1}$$, but then $$X_n$$ is forced. However, once we square each $$(X_i - \bar{X})$$, we lose the target value of $$0$$.

So how do we know that the degrees of freedom is $$n-1$$ without a target value? If there is no target value, why are we not able to make $$n$$ free choices?

• You have a target value, namely, the sample mean $\bar{X}$. – jbowman Jul 6 '19 at 21:35
• This has been answered before. See links in the right margin of this page under 'Related'. In particular, technical details are explored here. I continue this Comment in answer format with a simulation in R: to illustrate that there are $n-1$ degrees of freedom--not to prove the result. – BruceET Jul 6 '19 at 22:11
• @jbowman Are you able to elaborate on why $\bar{X}$ is a target value? As I see it, we've already freely chosen $X_1 \ldots X_n$ before we even compute $\bar{X}$. – Data Jul 7 '19 at 15:51
• You haven't freely chosen $n$ values of $(x_i-\bar{x})$, because the $x_i$ have to sum to $n\bar{x}$. Just as with your paragraph starting with "Suppose we choose..."... well, in your example, $\bar{x} = 10/3$. The sentence beginning "However, once we square each"... seems to imply you think that you don't have to pay attention to the fact that $\sum x_i/n = \bar{x}$ any more, but you do. – jbowman Jul 7 '19 at 17:24
• @jbowman Thank you for that explanation. I would like to know if I'm thinking about it clearly now. Suppose we choose $x_1=1, x_2=2, x_3=3$, so that $\bar{x} = \sum x_i /n = 2.$ In determining $\bar{x}$ we had $n$ degrees of freedom because I can choose any $n$ numbers I wanted. Now with the $(x_i - \bar{x})$, we can choose a new set of $x_i$, so long as we satisfy $\bar{x} = 2$. Hence we have a target. The sum $\sum (x_i - \bar{x})^2$ doesn't have a particular target, but the $x_i$ do. E.g. I can choose $x_1=1,x_2=1$, and to satisfy $\bar{x}=2$, I must choose $x_3=4$. Hence $n-1$ DOF. – Data Jul 7 '19 at 18:02

Consider samples of size $$n= 5$$ from a standard normal distribution. then $$Q =(n-1)S^2 \sim \mathsf{Chisq}(\nu=4),$$ not $$\mathsf{Chisq}(\nu=5).$$

set.seed(706);  m = 10^6;  n = 5
q = replicate( m, (n-1)*var(rnorm(n)) )
mean(q)
[1] 4.002257  # aprx E(Q) = 4
hdr = "Simulated Dist'n of Q fits CHISQ(4)[blue], not CHISQ(5) [red]"
hist(q, prob=T, br=50, col="skyblue2", main=hdr)

Note: If you consider $$n = 2,$$ then it is easy to verify that $$\bar X$$ is a function of $$X_1 + X_2$$ and independently, $$S^2$$ is a function of $$X_1 - X_2$$ so that $$S^2 \sim \mathsf{Chisq}(1).$$ One proof for $$n > 2$$ shows that $$\bar X$$ is a function of a vector in one dimention, and that (orthogonally) $$S^2$$ is a function of vectors in $$n-1$$ dimensions.