0
$\begingroup$

Target Matrix

Note: find_pct = ct_find / ct_rec

I want to use Fisher's exact test to decide if the apparent dip in find_pct of line 2 is real or due to random sampling. I apply Fisher's test using R to compare the ct_find:ct_rec for lines 1-2:

fisher.test(rbind(c(138,542-138), c(18,155-18)), alternative="less")

I obviously misunderstand how to use Fisher's, because, if I switch the order:

fisher.test(rbind(c(18,155-18), c(138,542-138)), alternative="less")

The results are completely different.

Improve this question
$\endgroup$
1
  • $\begingroup$ The 2 problems listed above are inverse of each other, the inverse of the the odds ratio of the second problem is equal to the odds ratio of the first problem. Depending on how you phase the the NULL hypothesis will be which form one takes. $\endgroup$
    – Dave2e
    Commented Jul 13, 2017 at 2:40

1 Answer 1

Reset to default
9
$\begingroup$

You have chosen to do a one-sided test and, obviously, order is important in a one-sided test. Your first call to fisher.test is testing the null hypothesis Pct1 = Pct2 vs the alternative that Pct1 < Pct2. The second call is testing the same null vs the alternative that Pct2 < Pct1. The two alternatives are opposites of one another, so they give p-values that are opposite, one being approximately 1 minus the other. The second call is the one that actually matches your research question.

In scientific research, one-sided tests are only done in special circumstances and have to be carefully justified. Usually you should do a two-sided test, by removing the alternative argument, in which case both of your calls would give the same p-value.

Also, incidentally, you've also chosen just one row out of 7 possible rows to test for a decreased percentage. This would appear to raise a multiple testing issue, unless you had a particular a priori reason for focusing on the second row.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Gordon, great point. To account for multiple testing, should I apply Bonferroni correction, and divide alpha (say, .05) by the number of rows (7), THEN conduct fishers on row1 v.s. row2, looking for a P-val < .05/7 ? $\endgroup$
    – Jesse Downing
    Commented Jul 20, 2017 at 15:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.