# Test equality of two measurements for a outcome

I have a binary outcome which were measured with two methods say A and B such a $2\times 2$ table like this:

> x<-matrix(c(349, 125, 474, 4, 7, 11, 353, 132, 485),3,3)
> dimnames(x)<-list(B=c("No", "Yes", "Total"),
+                                 A=c("No", "Yes", "Total"))
> x
A
B        No Yes Total
No    349   4   353
Yes   125   7   132
Total 474  11   485


from which, you can see the event rate for method A is 11/485=0.023 and for method B is 132/485=0.27. Intuitively, these two methods seem out of consistency. I did chisq.test with p.value 0.11 and a fisher exact test (since the rare events) with a p.value 0.01.

So, my question is which method should I use here to test: $H_0:$ these two methods are at equality for measure the outcome vs $H_1:$ they are different.

## 1 Answer

You should use the Fisher's exact test. You have cells with fewer than 5 (rule of thumb) and a 2x2 table so you can compute an exact p-value in this case rather than an approximate with a chi^2 test.

Conventionally, the condition applied (after Cochran, 1954) is that fewer than 20% (or 25% for others) of our expected frequencies should be less than 5, and none should be less than 1.


http://www.strath.ac.uk/aer/materials/4dataanalysisineducationalresearch/unit3/somefurtherthoughtsonusingchi-square/

• The key point is that the Null hypothesis of Fisher exact test is the not the Null hypothesis I would have. – David Z Oct 23 '14 at 16:41
• I see. You can use a t-test, you have two means and you want to test if they are different. H_0 : p_1 = p_2. Where p_1 and p_2 are your A and B above. – jimmyb Oct 23 '14 at 16:56