2
$\begingroup$

I don't find the powerfulness of Fisher's exact test on Wikipedia. Are there any results about the power of FET vs other methods on 2x2 contingency tables?

$\endgroup$

2 Answers 2

2
$\begingroup$

No Fisher's Exact Test is not the most powerful because it doesn't condition on the sufficient statistic, the odds ratio, of the likelihood. Pearson's chi-square test, on the other hand, is the score test which is asymptotically equivalent to the likelihood ratio test, and the LRT is uniformly most powerful (among unbiased tests). You can even show that the $\alpha$-level Fisher's Exact test results in sub-alpha false positive rates (it does not attain its nominal size).

This has been known for a long time. Bennett and Hsu (1960) is one of many papers discussing the point. There are many reasons to like the FET, but power is not one of them.

$\endgroup$
3
  • $\begingroup$ What are the reasons to like FET? $\endgroup$ May 9, 2019 at 19:35
  • $\begingroup$ @user1424739 because there's no issue of small sample size approximation: it uses the exact distribution of the possible permutations of the contingency table. $\endgroup$
    – AdamO
    May 9, 2019 at 21:24
  • 1
    $\begingroup$ Under the same conditions as used in the FET (i.e. conditioning on the margins), we can compute exact p-values for distribution of the Pearson chi-squared and the LRT from the permutations (or we can simulate tables to get p-values to any desired degree of accuracy). However, the LRT is only guaranteed to be UMP for a one sided test. $\endgroup$
    – Glen_b
    May 10, 2019 at 2:21
1
$\begingroup$

A computation of the power always depends on your assumptions. I recently uploaded a preprint to arxiv about the implementation of an unconditional test for contingency tables tentatively called m-test (link). It is work in progress, but we used a Monte Carlo simulation to compare the power of the m-test, FET and Barnard's test in R at different values of $\alpha$. The results with different marginals suggest that Barnard's test is more powerful than FET in most cases. The m-test was consistently the most powerful test of these three. There is one example here (the figures are at the end of the file, the Barnard package is verbose). The algorithm for the m-test is packaged here.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.