5
$\begingroup$

I attempted to understand the computation of the Fisher's exact test through programming using this formulation:

The Fisher exact test computes the p-values by finding the probabilities of all possible combinations of 2 × 2 tables that have the same marginal totals (the values in cells m, n, r, and s) that are equal to or more extreme that the ones observed.

Via enumeration of all different cell numbers, I can calculate the first part of the above formulation. This allows me to find all tables which have the same marginal totals. However, I encounter a problem with the second part, which involves finding equal or more extreme cases.

Test table:

A B $\sum$
C 8 7 15
D 11 3 14
$\sum$ 19 10 29

The number of tables with the same marginal totals is 11, including the above case.

$c_{1,1}$ $c_{1,2}$ $c_{2,1}$ $c_{2,2}$
5 10 14 0
6 9 13 1
7 8 12 2
8 7 11 3
9 6 10 4
10 5 9 5
11 4 8 6
12 3 7 7
13 2 6 8
14 1 5 9
15 0 4 10

Which of these are equal or more extreme? How can I identify them?

$\endgroup$
1
  • 2
    $\begingroup$ Good question. There's only one sensible answer for a one-tailed test - that is various sensible criteria all lead to the same answer -, but quite a few for a two-tailed test. Your choice might depend somewhat on the model for how the data are generated - e.g. comparing two binomial samples. $\endgroup$ Commented Mar 31 at 14:11

1 Answer 1

2
$\begingroup$

The null is that both A and B have the same proportion (of C's and D's). I.e. the 4 cells ought to be 9.83, 5.17, 9.17 and 4.83 (yes, it should only be integers, but that is what the null should look like, scaled to your total of 29). You are looking at the scenarios which are even further away from the null than what you measured. In what you measured, B has more C's than it should, and A has more D's than it should.

Formally, you are looking at the scenarios which have an equal or lower probability (under the null!) than what you observed.

To see if B really has more C's, the more extreme rows are the first 4 rows in your table. This adds up to .150 (hypergeometric distribution). That gives you a single sided Fisher test, for B having a higher proportion of C's.

For the double sided, you have to look at when A has even more D's (again, formally by comparing this probability to what you observed, conditioned on the marginal totals): it is more extreme if it is less than the probability of what you observed. These are the last 4 rows in your table of possibilities, adding up to 0.095. then, the single sided test (B>A) has a p-value of 0.150, the double sided (A<>B) has a p-value of .245.

One side is when B has even more C's, the other side is when A has even more D's, and you formally check that by computing the probability of all your possibilities (under the null, conditioned to the marginal totals) to what you observed: if that probability is lower, it is more extreme. And you used the hypergeometric distribution to evaluate all these probabilities.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.