This is a homework(-like) question. I know the answer but don't understand it.

If we were to answer the question "Are 5 different groups of mammals equally common?", what test would you perform?

The answer is: goodness-of-fit test to a Chi-squared distribution.


Why a Chi-square distribution? I would rather chose a null hypothesis to be that all group are equally common, therefore the abundance would be uniformally distributed. I would calculate a chi-squared value using $\frac{N}{5}$ as expected values (for the 5 expected values), where $N$ is the total number of observed individuals. Wouldn't it be right? Therefore, I would do a g-o-f test to a uniform distribution, isn't it?

  • $\begingroup$ You're right about the null hypothesis--but exactly how would you go about testing it? $\endgroup$ – whuber Nov 24 '14 at 18:41
  • $\begingroup$ @whuber I would calculate a chi-squared value using $\frac{N}{5}$ as expected values (for the 5 expected values), where $N$ is the total number of observed individuals. Wouldn't it be right? Therefore, I would do a g-o-f test to a uniform distribution, isn't it? Thanks! $\endgroup$ – Sulawesi Nov 24 '14 at 18:43
  • $\begingroup$ What's your question, then? With the edit it asks why a $\chi^2$ GOF test should be used and then explains why a $\chi^2$ GOF test should be used. There seems to be nothing left to say. $\endgroup$ – whuber Nov 24 '14 at 18:51
  • $\begingroup$ oh sorry if it was unclear. The question is why a goodness-of-fit test to a chi-squared distribution? Maybe the question here makes more sense because the distribution is not chi-squared but binomial. Let me know if my question makes sense. I may just missunderstand the way the answer key is formulated! $\endgroup$ – Sulawesi Nov 24 '14 at 18:56
  • $\begingroup$ A related question with more focus on why chi-squared goodness of fit test can be used to test if data come from the binomial distribution: stats.stackexchange.com/questions/104468/… $\endgroup$ – Silverfish Nov 28 '14 at 0:13

There are two different distributions in play - the distribution of your data (or strictly speaking, the hypothesised distribution of the population they are drawn from), and the distribution of your test statistic under the assumption that the null hypothesis is correct. This is a little subtlety that I find can catch people out when they are new to the concept, but in fact it is usually the case that the null distribution of your test statistic is quite different to the (hypothesised or actual) distribution of your data.*

Using a chi squared goodness of fit test means that if your null hypothesis is true, then your test statistic, $\chi^2 = \sum_{i=1}^n {\frac{(O_i - E_i)}{E_i}^2}$, would follow a chi square distribution (at least, approximately - you may hear it called "asymptotic", i.e. for large samples it should be close enough to chi square for practical purposes). This is why you use the chi square distribution tables. Remember that $\chi^2$ is larger if your data is a bad fit to the hypothesised distribution, since in this case the square of the difference between expected and observed frequencies, the $(O_i - E_i)^2$ in the numerators, is large. So if $\chi^2$ is larger than the critical value in the tables, you have significant evidence against the null hypothesis, in the sense that a $\chi^2$ value that large, and hence a fit that poor, would be unlikely if the hypothesised model were true.

None of this means that your data is chi squared distributed, or that you expect it to be so. The expected frequencies $E_i$ used in the $\chi^2$ calculation are premised on the population having whatever distribution is specified according to $H_0$.

We can look at this in more detail. Supposing that $H_0$ is true so the expected frequencies are "correct" (they may not match the observed frequencies precisely, but if we were to take hundreds of samples and tabulate each one, then in each category the average of our observed frequencies should be very close to the expected one), then the quantity $\frac{O_i - E_i}{\sqrt{E_i}}$ will behave approximately like a z-score for that cell. The sum over all the cells of $\sum_{i=1}^n {\frac{(O_i - E_i)}{E_i}^2}$ is then, roughly, the sum of squared z-scores. Perhaps you already know that the random chi-squared variable with $\nu$ degrees of freedom, $\chi^2_\nu$, is the sum of the squares of $\nu$ independent standard normal variables. You may now be able to see that, so long as your data were drawn from a population whose distribution matched your null model, your test statistic will approximately follow a chi-squared distribution.

Things are actually slightly more complicated - since the total of the expected frequencies equals the total of the observed frequencies, the total of $O_i - E_i$ must be zero. Hence the value of $O_i - E_i$ in the last cell is completely determined by what happened in the previous cells, so our "z-scores" weren't quite independent. Fortunately we can compensate for this by subtracting one from the degrees of freedom, which explains why in your case, with 5 cells in your table and hence 5 values of $\frac{(O_i - E_i)^2}{E_i}$ in your sum, you'd compare your test statistic to the critical value listed in the tables for $\chi^2_4$. If your test statistic is above the critical value, this tells you the sum of squared z-scores would be unlikely to be so high. Were your null hypothesis true, then your test statistic should behave like the sum of squared z-scores, so the fact it came out so high constitutes evidence against the null hypothesis - i.e. it suggests your population did not follow the hypothesised distribution.

$*$ The very first hypothesis test that many people learn is the Z test for the mean, where the data are sampled from a normal distribution of known variance $\sigma^2$ and the null hypothesis is $H_0: \mu = \mu_0$ . In this case, all three of the population distribution, the sample mean and the z-score (the test statistic for this test) are normally distributed. But this is rarely true for hypothesis tests in general. Moreover, for the Z test, assuming the null hypothesis is true, their distributions are $X \sim \mathcal{N}(\mu_0, \sigma^2)$, $\bar{X} \sim \mathcal{N}(\mu_0, \frac{\sigma^2}{n})$ and $Z \sim \mathcal{N}(0, 1)$. So on closer inspection we see that, even here, the distributions of the data and the test statistic are quite different after all.

  • 2
    $\begingroup$ +1 It might help your readers' understanding a little to point out that the test statistic only approximately has a $\chi^2$ distribution. Indeed, the test statistic can be seen as a way of combining five $Z$ statistics of the form $(O_i-E_i)/\sqrt{E_i}$, one for each group, into a single overall statistic reflecting all five $Z$'s at once. Thus a glimmer of the connection to the Normal theory in your last paragraph may show through the otherwise mysterious formula. $\endgroup$ – whuber Nov 24 '14 at 21:13
  • 1
    $\begingroup$ @whuber I was tempted to do so, I might expound on it yet, but hopefully your comment provides some insight at least. I regularly teach this topic to high school students and I find the biggest thing that throws them is that there are two completely different distributions in play, each serving a completely different purpose. If I've managed to communicate clearly why the two are distinct, I've achieved the main aim! $\endgroup$ – Silverfish Nov 24 '14 at 21:20
  • 1
    $\begingroup$ @whuber Right, I added a bit. A bit of hand-waving about degrees of freedom but I think at the introductory level this is probably the best approach! For the benefit of future readers, do we already have some good Q&As on the degrees of freedom adjustment that could be linked in the answer? $\endgroup$ – Silverfish Nov 24 '14 at 22:53
  • 1
    $\begingroup$ Check out stats.stackexchange.com/questions/16921/…. $\endgroup$ – whuber Nov 24 '14 at 22:59
  • $\begingroup$ @whuber That is one of my all-time favourite threads on CV. Maybe a little daunting for a beginner, but very valuable! As a more specific supplement, do we have a thread that addresses the issue of df for a chi square goodness of fit test? Preferably in the basic case like this one, though the meatier problem where some parameters of the fitted model were estimated from the data would do. I'm thinking along the lines of "a next step for the curious". $\endgroup$ – Silverfish Nov 24 '14 at 23:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.