Consider an $m \times n$ contingency table and test of independence.

Suppose the expected counts are such that chi-squared test is not recommended (close to zero in some cells), as it becomes pretty inaccurate.

Instead, we can run exact Fisher test.

Question 1: What are the options to compute a power of exact Fisher test? Is there a way without running expensive simulations?

Question 2 (pretty hypothetical): How bad would it be to compute a power of a chi-squared test as a proxy for power of exact Fisher test? Computationally, it's really fast, involves effect size (closed form expression of the table entries), sample size, dofs, and alpha (p-value threshold). But is it a reasonable approximation of the true power?

  • $\begingroup$ The test statistic for Fisher's exact test has a hypergeometric distribution. Various implementations make different choices determining P-values for two-sided tests, so finding power will be more straightforward if you have a one-sided alternative. Of course, to find 'power' you need to have a specific alternative in mind. A complication in comparing Fisher with chi-sq is that chi-sq tests are usually two-sided. // So for a specific answer, you need to settle on a specific alternative and on a one vs. a two-sided test. $\endgroup$ – BruceET Sep 14 '20 at 21:14
  • $\begingroup$ Good point. I was thinking of a two-sided both for Fisher and chi sq. For Fisher itself I'm using implementation equivalent to fisher.test from R. For chi sq. the alternative would be a non-central chi sq. distribution. $\endgroup$ – max Sep 14 '20 at 23:06
  • $\begingroup$ There is a power and sample size procedure in R for the Fisher exact test, but not in a library I currently have installed. Google power Fisher exact test---and look past the ads. $\endgroup$ – BruceET Sep 14 '20 at 23:23
  • $\begingroup$ @max: Is this for $m=n=2$, where there are multiple packages or is it for general $m\times n$ tables, which is considerably more difficult? $\endgroup$ – Thomas Lumley Sep 14 '20 at 23:50
  • $\begingroup$ The question is for general $m$ and $n$, my particular table is $2 \times 6$. The R rdocumentation.org/packages/statmod/versions/1.4.34/topics/… function seems to only support 2x2 tables. (I'm actually calling R function from python, which is already a stretch, but somehow python doesn't seem to have exact Fisher test for large tables). $\endgroup$ – max Sep 15 '20 at 0:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.