2
$\begingroup$

On Wikipedia, it says

If $X_{i};i=1,\ldots ,n$ are independent normal $(\mu ,\sigma ^{2})$ random variables, the statistic

$$\frac{\sum\limits^n_{i=1}(X_i-\bar{X})^2}{\sigma^2}$$

follows a chi-squared distribution with $n − 1$ degrees of freedom. Here, the degrees of freedom arises from the residual sum-of-squares in the numerator, and in turn the $n − 1$ degrees of freedom of the underlying residual vector $\{X_{i}-{\bar {X}}\}$.

With that in mind, I'd like to put forward the following example:

Suppose we choose an integer, $10$. Let's say our goal is to choose some $n$ values that sum to $10$. Let's say $n=3$. I freely choose $6$ and then $3$, but now I am forced to choose $1$. Hence I have $2$ degrees of freedom. Or more generally, $n-1$ degrees of freedom.

In this example, it's clear that the final choice is not a free one, because we have some target value, and so once we've made $n-1$ choices for the summands, there is only one possible value that ensures we satisfy the target sum value constraint.

Now, going back to case of the residual sum-of-squares on the numerator. If this were $\sum\limits^n_{i=1}(X_i-\bar{X})$ instead (notice I've dropped the square), then our target is $0$. So again, like in the example above, we have $n-1$ degrees of freedom because we can make free choices for $X_1\ldots X_{n-1}$, but then $X_n$ is forced. However, once we square each $(X_i - \bar{X})$, we lose the target value of $0$.

So how do we know that the degrees of freedom is $n-1$ without a target value? If there is no target value, why are we not able to make $n$ free choices?

$\endgroup$
  • $\begingroup$ You have a target value, namely, the sample mean $\bar{X}$. $\endgroup$ – jbowman Jul 6 at 21:35
  • $\begingroup$ This has been answered before. See links in the right margin of this page under 'Related'. In particular, technical details are explored here. I continue this Comment in answer format with a simulation in R: to illustrate that there are $n-1$ degrees of freedom--not to prove the result. $\endgroup$ – BruceET Jul 6 at 22:11
  • $\begingroup$ @jbowman Are you able to elaborate on why $\bar{X}$ is a target value? As I see it, we've already freely chosen $X_1 \ldots X_n$ before we even compute $\bar{X}$. $\endgroup$ – HumptyDumpty Jul 7 at 15:51
  • $\begingroup$ You haven't freely chosen $n$ values of $(x_i-\bar{x})$, because the $x_i$ have to sum to $n\bar{x}$. Just as with your paragraph starting with "Suppose we choose..."... well, in your example, $\bar{x} = 10/3$. The sentence beginning "However, once we square each"... seems to imply you think that you don't have to pay attention to the fact that $\sum x_i/n = \bar{x}$ any more, but you do. $\endgroup$ – jbowman Jul 7 at 17:24
  • $\begingroup$ @jbowman Thank you for that explanation. I would like to know if I'm thinking about it clearly now. Suppose we choose $x_1=1, x_2=2, x_3=3$, so that $\bar{x} = \sum x_i /n = 2.$ In determining $\bar{x}$ we had $n$ degrees of freedom because I can choose any $n$ numbers I wanted. Now with the $(x_i - \bar{x})$, we can choose a new set of $x_i$, so long as we satisfy $\bar{x} = 2$. Hence we have a target. The sum $\sum (x_i - \bar{x})^2$ doesn't have a particular target, but the $x_i$ do. E.g. I can choose $x_1=1,x_2=1$, and to satisfy $\bar{x}=2$, I must choose $x_3=4$. Hence $n-1$ DOF. $\endgroup$ – HumptyDumpty Jul 7 at 18:02
5
$\begingroup$

Consider samples of size $n= 5$ from a standard normal distribution. then $Q =(n-1)S^2 \sim \mathsf{Chisq}(\nu=4),$ not $\mathsf{Chisq}(\nu=5).$

set.seed(706);  m = 10^6;  n = 5
q = replicate( m, (n-1)*var(rnorm(n)) )
mean(q)
[1] 4.002257  # aprx E(Q) = 4
hdr = "Simulated Dist'n of Q fits CHISQ(4)[blue], not CHISQ(5) [red]"
hist(q, prob=T, br=50, col="skyblue2", main=hdr)
  curve(dchisq(x, 4), add=T, lwd=2, col="blue")
  curve(dchisq(x, 5), add=T, lwd=3, col="red", lty="dotted")

enter image description here

Note: If you consider $n = 2,$ then it is easy to verify that $\bar X$ is a function of $X_1 + X_2$ and independently, $S^2$ is a function of $X_1 - X_2$ so that $S^2 \sim \mathsf{Chisq}(1).$ One proof for $n > 2$ shows that $\bar X$ is a function of a vector in one dimention, and that (orthogonally) $S^2$ is a function of vectors in $n-1$ dimensions.

$\endgroup$
  • 1
    $\begingroup$ The note does it. $\endgroup$ – Sextus Empiricus Jul 7 at 0:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.