I have a binary outcome which were measured with two methods say A and B such a $2\times 2$ table like this:

> x<-matrix(c(349, 125, 474, 4, 7, 11, 353, 132, 485),3,3)
> dimnames(x)<-list(B=c("No", "Yes", "Total"),
+                                 A=c("No", "Yes", "Total"))
> x
B        No Yes Total
  No    349   4   353
  Yes   125   7   132
  Total 474  11   485

from which, you can see the event rate for method A is 11/485=0.023 and for method B is 132/485=0.27. Intuitively, these two methods seem out of consistency. I did chisq.test with p.value 0.11 and a fisher exact test (since the rare events) with a p.value 0.01.

So, my question is which method should I use here to test: $H_0:$ these two methods are at equality for measure the outcome vs $H_1:$ they are different.


1 Answer 1


You should use the Fisher's exact test. You have cells with fewer than 5 (rule of thumb) and a 2x2 table so you can compute an exact p-value in this case rather than an approximate with a chi^2 test.

Conventionally, the condition applied (after Cochran, 1954) is that fewer than 20% (or 25% for others) of our expected frequencies should be less than 5, and none should be less than 1. 


  • $\begingroup$ The key point is that the Null hypothesis of Fisher exact test is the not the Null hypothesis I would have. $\endgroup$
    – David Z
    Oct 23, 2014 at 16:41
  • $\begingroup$ I see. You can use a t-test, you have two means and you want to test if they are different. H_0 : p_1 = p_2. Where p_1 and p_2 are your A and B above. $\endgroup$
    – jimmyb
    Oct 23, 2014 at 16:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.