3
$\begingroup$

The Wikipedia page on Pearson's chi-squared test states that a difference to Fisher's exact test is that the latter makes the "assumption of fixed marginal distributions". I assume that applies to the use of chi-squared as a test of independence, because that is what Fisher's exact test does.

However, the description of the chi-squared test for independence clearly shows that it is a goodness-of-fit test which compares the observed joint distribution $O_{i,j}$ to the one expected under independence from the marginal distributions, $$ E_{i,j} = N p_{i\cdot} p_{\cdot j}, $$ which in turn are estimated from the observed marginals, $$ p_{i\cdot} = \frac{O_{i\cdot}}{N} = \frac1N \sum_j O_{ij}, $$ and correspondingly for $p_{\cdot j}$. As far as I can see that means that the test is conditional on given $N$, $p_{i\cdot}$, and $p_{\cdot j}$.

Can someone clear this up? Maybe Wikipedia is simply wrong?

$\endgroup$

1 Answer 1

2
$\begingroup$

The "exactness" of Fisher's test depends on the assumption that each of the row totals and column totals (the "margins") is given as part of the experimental design. That's what's meant by "fixed marginal distributions." In that case the distribution of counts among cells exactly follows a hypergeometric distribution.

For example, the alleged origin of the test was to examine whether Dr. Muriel Bristol could distinguish cups of tea in which the milk had been added first from those in which the tea had been added first. The experimenters specified how many cups of each type had been prepared, and Dr. Bristol had to guess the same number of cups of each type overall. Thus both the row total and column totals (cups of each type prepared, cups of each type reported) were fixed by design. This page and this page provide further information and links.

With the chi-square test, in contrast, only the total number of counts, $N$, is taken as pre-specified. Although you do observe particular row totals and column totals, those are not fixed by design. Unlike with pre-specified row and column totals, the statistical test must take into account the sampling variability that might have led to those observed row and column totals, given only the toal number of counts.

In practice, Fisher's test is often applied in situations where the row and column totals weren't fixed by design. That's generally taken to be OK, as the test is then typically more conservative (less likely to report false positives) than alternate tests like chi-square.

$\endgroup$
2
  • 1
    $\begingroup$ (1) Pearson's test is also asymptotically valid for models with no, either, or both sets of marginal totals fixed. (2) The use of Fisher's Exact Test when marginal totals aren't fixed is often positively argued for - see When and how to avoid inappropriate use of Fisher's exact test & links. $\endgroup$ Dec 17, 2020 at 8:40
  • $\begingroup$ Fisher himself did not assume the marginals were fixed by the design (although it were in the urexperiment involving Dr. Muriel Bristol tasting tea). He argued that the marginals were ancillary statistics and so could be conditioned upon. (that have been shown in general not to be exactly true). $\endgroup$ Feb 5, 2021 at 20:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.