I don't find the powerfulness of Fisher's exact test on Wikipedia. Are there any results about the power of FET vs other methods on 2x2 contingency tables?


No Fisher's Exact Test is not the most powerful because it doesn't condition on the sufficient statistic, the odds ratio, of the likelihood. Pearson's chi-square test, on the other hand, is the score test which is asymptotically equivalent to the likelihood ratio test, and the LRT is uniformly most powerful (among unbiased tests). You can even show that the $\alpha$-level Fisher's Exact test results in sub-alpha false positive rates (it does not attain its nominal size).

This has been known for a long time. Bennett's paper from 1960 is one of many discussing the point


There are many reasons to like the FET, but power is not one of them.

  • $\begingroup$ What are the reasons to like FET? $\endgroup$ May 9 '19 at 19:35
  • $\begingroup$ @user1424739 because there's no issue of small sample size approximation: it uses the exact distribution of the possible permutations of the contingency table. $\endgroup$
    – AdamO
    May 9 '19 at 21:24
  • $\begingroup$ Under the same conditions as used in the FET (i.e. conditioning on the margins), we can compute exact p-values for distribution of the Pearson chi-squared and the LRT from the permutations (or we can simulate tables to get p-values to any desired degree of accuracy). However, the LRT is only guaranteed to be UMP for a one sided test. $\endgroup$
    – Glen_b
    May 10 '19 at 2:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.