18
$\begingroup$

As this source states, one of the assumptions to perform Fisher's exact test of independence is that the row and column totals should be fixed. However, I find the explanation coming with it pretty vague and I couldn't find any examples illustrating the principle on the internet.

Would anyone have a good example explaining this assumption?

$\endgroup$

1 Answer 1

23
$\begingroup$

In my opinion, the source that you link to is wrong in that it is confusing conditioning with assumptions.

Fisher's exact test conditions on the margin totals, meaning that it does not use any information about independence that might be inferable from the margin totals. In probability theory, once you condition on a random variable, the random variable is then treated as fixed in the downstream computations. This however is purely a mathematical device. It is not at all the same as assuming that the margin totals were fixed in advance as part of the experimental design. It is simply a matter of choosing what information to extract from the data.

Conditioning actually reduces the assumptions made by the test. Since Fisher's test only uses the distribution of the cell counts given the margin counts, it therefore makes no assumptions about how the margin counts were generated.

The source you link to states that Fisher's exact test makes more assumptions than Pearson's chisquare test or the G-test. In my opinion, this statement is wrong. All three tests condition on the margin totals and all three make the same distributional assumptions. In the case of Fisher's test, the conditioning is explicit. For the other two tests, the conditioning is implicit from the way that the nuisance parameters are set but is nevertheless real. Fisher's exact test gives correct control of the type I error rate regardless of whether the marginal totals are fixed or not and the p-values are exact in the sense that they are derived from exact computations using the hypergeometric distribution.

IMO there is a lot of misunderstanding about this issue on the internet and it possibly derives from the question of optimality. If the margin totals are not fixed by the experimental design, then potentially there is information about independence contained in the row and column totals, and hence potentially one might be able to create a statistical test that is more powerful than Fisher's exact test. There is some interesting literature about this but in practice (i) there's not much information to retrieve if the counts are small and (ii) Fisher's exact test is already powerful enough when the counts are large.

Another misunderstanding is that people interpret "exact" to mean that the type I error rate is controlled exactly at a specified rate. Like all tests that return exact p-values for discrete counts, Fisher's test gives somewhat conservative control of any pre-specified type I error rate, simply because the p-values are granular (and this has nothing to do with whether the margins are fixed or not).

$\endgroup$
2
  • 1
    $\begingroup$ Could you maybe provide some links for your statements? There seems to be lots of sources stating that Fisher's test requires the rare condition that both row and column marginal totals are fixed in advance, e.g. here. but if this is not the case I would like some literature to point to. $\endgroup$
    – A. Bollans
    Commented Dec 27, 2023 at 16:55
  • $\begingroup$ @A.Bollans One suitable reference would be F. Yates, "Tests of Significance for 2x2 Contingency Tables", Journal of the Royal Statistical Society. Series A (General), Vol. 147, No. 3 (1984), pp. 426-463. This article is particularly good because it points that that Fisher's original "tea tasting" example did in fact have fixed margins, a choice that Fisher made for pedagogic clarity, but which has in good part caused the whole confusion about requiring fixed margins. $\endgroup$ Commented Feb 26 at 0:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.