Fisher's Exact Test in contingency tables larger than 2x2

I was taught to only apply Fisher's Exact Test in contingency tables that were 2x2.

Questions:

1. Did Fisher himself ever envision this test to be used in tables larger than 2x2 (I am aware of the tale of him devising the test while trying to guess whether an old woman could tell if milk was added to tea or tea was added to milk)

2. Stata allows me to use Fisher's exact test to any contingency table. Is this valid?

3. Is it preferable to use FET when expected cell counts in a contingency table are < 5?

• You might want to consider how good Fisher's exact test really is: stat.columbia.edu/~gelman/research/published/isr.pdf (Section 3.3)
– Fr.
Dec 8, 2012 at 13:32
• You can compute the Fisher's exact test in R. Their method actually uses a network algorithm to make it fast enough: stat.ethz.ch/R-manual/R-patched/library/stats/html/… See the paper [Mehta and Patel 1986] Feb 19, 2015 at 2:22
• @Fr. Thanks for that informative link. Are you suggesting that Fisher's exact test is actually not that good? Aug 4, 2020 at 15:27
• @don.joey Well, what do you make of the ref. I posted?
– Fr.
Aug 15, 2020 at 13:06
• @don.joey Apologies if the ref. is not explicit enough. In substance, it blames Fisher's exact test with being neither exact, nor a test (in the sense of: something that returns a p-value that you can interpret like the p-values that you get from e.g. an F-test).
– Fr.
Aug 29, 2020 at 12:08

The only problem with applying Fisher's exact test to tables larger than 2x2 is that the calculations become much more difficult to do. The 2x2 version is the only one which is even feasible by hand, and so I doubt that Fisher ever imagined the test in larger tables because the computations would have been beyond anything he would have envisaged.

Nevertheless, the test can be applied to any mxn table and some software including Stata and SPSS provide the facility. Even so, the calculation is often approximated using a Monte Carlo approach.

Yes, if the expected cell counts are small, it is better to use an exact test as the chi-squared test is no longer a good approximation in such cases.

• Can you point me where I can find documentation on how to do the Fisher test using SPSS? Thanks May 20, 2011 at 2:48
• Look under crosstabs May 20, 2011 at 5:57
• Given that software can do the calculation so easily nowadays, is there any circumstance where, theoretically or practically, Chi squared test is actually preferable to Fisher's exact test? Jul 9, 2011 at 11:36
• In many cases the "exact" test is not exact at all and many "approximate" methods have a coverage closer to the nominal level of significance. See e.g. Alan Agresti and Brent A. Coull (1998) "Approximate Is Better than "Exact" for Interval Estimation of Binomial Proportions" The American Statistician, 52(2):119-126. Apr 22, 2013 at 8:15
• What is roughly the maximum contingency table size that can be analyzed with Fisher's exact method in a feasible time frame (say a week on a standard laptop)?
– pir
Apr 19, 2016 at 17:04

This page in MathWorld explains how the calculations work. It points out that the test can be defined in a variety of ways:

To compute the P-value of the test, the tables must be ordered by some criterion that measures dependence, and those tables that represent equal or greater deviation from independence than the observed table are the ones whose probabilities are added together. There are a variety of criteria that can be used to measure dependence.

I have not been able to find other articles or texts that explain how this is done with tables larger than 2x2.

This calculator computes the exact Fisher's test for tables with 2 columns and up to 5 rows. The criterion it uses is the hypergeometric probability of each table. The overall P value is the sum of the hypergeometric probability of all tables with the same marginal totals whose probabilities are less than or equal to the probability computed from the actual data.

If you're looking for other ways to compute Fisher's exact test with larger contingency tables, here is a online calculator for Fisher's exact test for 2x3 contingency tables. Also, here's one for 3x3 contingency tables, and one for 2x4 contingency tables.

Yes, if the expected cell counts are small, it is better to use Fisher's exact test instead of the chi-squared test, if possible.

In order to obtain Fisher"s Exact Test in SPSS, use the Statistics = Exact option in Crosstabs. Methods for computing the Exact Tedt for larger tables have been around at least since the 1960"s. The speed of modern microprocessors makes the computation time inconsequential these days. Indeed, it is so easy to run the Exact Test that it is important not to use it too widely.

One important thing to keep in mind here is that Fisher's exact test is typically implemented for contingency tables with fixed margins, i.e. the efficient algorithms utilized in Stata and R involve either generating all tables with fixed margins or sampling all tables with fixed margins.

However, the assumption of fixed margins is not appropriate in every case. In fact, I think Agresti argues that it is rarely appropriate though this opinion is debated. In any case, before you utilize Fisher's exact test as it's commonly implemented, you need to think about whether it's appropriate for your application to treat both row and column sums as fixed.

• The argument for conditioning on the margins is that the margins are approximately ancillary, that is, their distribution do not depend on the parameters. Then Fishers exact test is a case of conditional (on ancillaries) frequentist inference! Feb 13 at 16:01