The fisher.test function in base R by default returns a confidence interval for the odds ratio in a 2x2 contingency table. For example:

> x <- c(100, 5, 70, 12)
> dim(x) <- c(2,2)
> fisher.test(x)

    Fishers Exact Test for Count Data

data:  x
p-value = 0.02291
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
  1.058526 12.904604
sample estimates:
odds ratio 

The confidence interval of an odds ratio is an extremely useful thing to know, and I would like to refer to it in an article I am currently writing. My dataset has high enough n for a chi-square test, but the latter would only give me the test statistic and a p-value, which are harder to interpret than the confidence interval of an odds ratio. However, I cannot find any explanation of how the confidence interval is being calculated in this case, nor of what the theoretical precedent might be for calculating confidence intervals of odds ratios as part of a Fisher test (as opposed to a logistic regression).

Can anyone shed some light?

  • 1
    $\begingroup$ I think there's other posts on this here, but basically it's based on the distribution of possible permutations of the contingency tables, having the same marginal frequencies, and the resulting odds ratio of the effect, excluding the extreme symmetric 2.5% of tables on either tail of effect. $\endgroup$
    – AdamO
    May 20 '19 at 14:23
  • $\begingroup$ Thank you, @AdamO. If there are other posts on this here, could you link to them? I haven't been able to find any. (Also if you could provide a source for your answer, that would be ideal - but of course there may be sources provided by the other posts on this topic, in which case that's not necessary.) $\endgroup$ May 20 '19 at 15:29

The R help manual cites the Fisher letter to the Australian Journal of Statistics.

In it he notes, by example:

If the observations in a $2 \times 2$ table are distinctly out of proportion (and indeed in other cases also) we may wish to set limits to the true product ratio, e.g. the observed table

$$ \begin{array}{cc} 10 & 3 \\ 2 & 15 \end{array}$$

gives a crude ratio of 25. How small could the true ratio be in reasonable consistency with the data? If the expectation in the four classes were

$$ \begin{array}{cc} 10-x & 3+x \\ 2+x & 15-x \end{array}$$

the true ratio would be $(10-x)(15-x)/(3+x)(2+x)$m and $\chi^2$ for the observations would be:

$$\chi^2 = x^2 \left( \frac{1}{10-x} + \frac{1}{3+x} + \frac{1}{2+x} + \frac{1}{15-x} \right)$$

so if $x$ were 3.0, $$\chi^2 = 3^2 (0.59286) = 5.3357$$ with one degree of freedom.

The exact probability of such a small sample of 30 giving 10 or more in the first quadrant is the partial sum of a hypergeometric series, and not easy to calculate for if $\xi$ stand for the theoretical product ratio, the frequencies of 0 to 12 in the quadrant will be proportional to the terms:

$$ 1, \frac{13 \times 12}{1\times 6}\xi, \frac{13\times 12 \times 12 \times 11}{1 \times 2 \times 6 \times 7}\xi^2, \ldots, \frac{13!12!5!}{(13-r)!(12-r)!(5+r)!}\xi^i,\ldots$$

It would not be too difficult, as in the exact test for disproportionality, to calcuate the last three terms for any chosen value of $\xi$, but for the ratio of these to the whole we would require the sum of the entire series or $$F(-13, -12, 6, \xi)$$ which would be best obtained by calculating all the terms and summing them, a process too lengthy to be recommended.

Using Yates' adjustment, however, we can at once find: $$\chi^2_c = (2.5)^2 0.59286 = 3.7054$$.

Further taking $x=3.1$ we have

$$ \chi^2_c = (2.6)^2(0.58717) = 3.9693$$

Interpolating for the tabular entry 3.841 it appears that $x=3.0501$ and the cross product ratio is 2.718.

So that it may be inferred from the data that the true cross-product ratio exceeds 2.718 unless a coincidence of one in forty has occurred, Similar limits can be set in both directions and at all limits of probability.

  • $\begingroup$ Okay thanks! So presumably this is the method being used and the theoretical justification for it? (It would have been nice if they'd given an explanation of that beyond a citation of a paywalled letter but hey...) $\endgroup$ May 20 '19 at 15:56
  • $\begingroup$ @Westcroft_to_Apse Yes, basically. Agreed at the frustration of paywalled research. R's authors are academics, and when it came to citing other research, not only were they unburdened by paywalls, but they strove to cite original research when implementing the language. That makes learning theory from R a bit of a paradox: there are "open learning" tools to teach you the same methods, but you need the old research to actually know they are the same (and some open learning tools are garbage, so buyer beware). $\endgroup$
    – AdamO
    May 20 '19 at 16:22
  • $\begingroup$ It’s not just the frustration of paywalled research, it’s the laziness of citing something rather than taking a few minutes to write a couple of sentences to explain it. (Which is kind of crazy given how much longer it must have taken to implement this in FORTRAN than it would have taken to write those sentences and copy out Fisher’s equations.) But this function has been part of R for a long time, and I guess that standards of documentation were different in those days. Thanks again! $\endgroup$ May 23 '19 at 8:15
  • 1
    $\begingroup$ @Westcroft_to_Apse well such questions are always on topic for CV. Ask liberally. This site could be an excellent companion to learn statistics through R for curious readers. $\endgroup$
    – AdamO
    May 23 '19 at 14:29
  • $\begingroup$ Could you possibly help me out with some advice on exactly that? $\endgroup$ Jun 14 '19 at 21:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.