0
$\begingroup$

I have a 2x2 contingency table and i want to calculate if the pair inside is significantly different. i made a matrix like the following named raw_matrix

          CNS random
Not_H3K4  343  28825
H3K4      11   2014

Create this matrix , thus:

raw_matrix = structure(c(343, 11, 28825, 2014), 
    .Dim = c(2L, 2L), .Dimnames = list(
    c("NotH3K", "H3K"), c("CNS", "Random")))

as i searched, unconditional exact test like Barnard’s and Boschloo’s exact tests are the most powerful test for this end. i installed the 'Exact' package and tried to do the test using this command:

exact.test(raw_matrix)

it took more than half an hour on a 64GB ram and 3.5 GH CPU computer and finally it gave the following error:

    Error: cannot allocate vector of size 42.0 Gb
In addition: Warning messages:
1: In matrix(A[xTbls + 1, ] * B[yTbls + 1, ], ncol = length(int)) :
  Reached total allocation of 61417Mb: see help(memory.size)
2: In matrix(A[xTbls + 1, ] * B[yTbls + 1, ], ncol = length(int)) :
  Reached total allocation of 61417Mb: see help(memory.size)
3: In matrix(A[xTbls + 1, ] * B[yTbls + 1, ], ncol = length(int)) :
  Reached total allocation of 61417Mb: see help(memory.size)
4: In matrix(A[xTbls + 1, ] * B[yTbls + 1, ], ncol = length(int)) :
  Reached total allocation of 61417Mb: see help(memory.size)

then i installed 'Exact2x2' package and did the test using this command:

exact2x2(raw_matrix)

which gave me the following results:

    Two-sided Fisher's Exact Test (usual method using minimum likelihood)

data:  raw_matrix
p-value = 0.006433
alternative hypothesis: true odds ratio is not equal to 1
95 percent confidence interval:
 1.2028 4.2424
sample estimates:
odds ratio 
  2.178631 

but as i read in the 'Exact'package tutorial , the fisher exact test which is a conditional exact test is not so powerful. finally i did the normal chi square test using the command chisq.test(raw.matrix) which gave the following results that is different from fisher test's results:

    Pearson's Chi-squared test with Yates' continuity correction

data:  test_1
X-squared = 6.2045, df = 1, p-value = 0.01274

im a Geneticist and not an expert in statistics, i appreciate if anybody could tell me what is the best strategy here to do this test

$\endgroup$
3
  • $\begingroup$ It seems you can reject the null hypothesis of no association based on Fisher's exact test. Why bother about power? $\endgroup$ – Michael M Aug 15 '14 at 15:25
  • $\begingroup$ Depends what margins were fixed in the sampling scheme & whether you want to make arguments from (approximate) ancillarity to justify conditioning on those that weren't. Comparison of unconditional vs conditional power is not meaningful. $\endgroup$ – Scortchi - Reinstate Monica Aug 15 '14 at 15:39
  • $\begingroup$ thanks for answer. as i mentioned , im not an expert in statistics, i was just trying to look for the best method to do the test.in the exact method tutorial[cran.r-project.org/web/packages/Exact/Exact.pdf] it says that "Unconditional tests (such as Barnard’s and Boschloo’s exact tests) are more powerful alternatives than conditional tests (such as Fisher’s exact test)." but now based on Eric brady's answer it seems that a fisher test would be powerful enough for this analysis. $\endgroup$ – user3015703 Aug 17 '14 at 4:33
1
$\begingroup$

What is the nature of you're underlying data? It could be the case that the approximation provided by the Chi-Squared test is reasonable. The basic idea is that if you have enough data, and it is reasonably evenly distributed across the cells in your table, the Chi-Squared approximation is reasonable (as long as other assumptions are met such as random sampling). The general rule of thumb given is that each cell should have at least 80% of the cells have a count of 5 or greater, and no cells have a count of 0. This is a heuristic, so if you have very unbalanced data or something like that you might want to do a bit more research, but if appropriate conditions are satisfied you may just want to proceed with a Chi-Squared test.

If these criterion are not met, Fisher showed that p-values for 2x2 tables can be obtained exactly as the probability of the cell counts can be shown to be a Hypergeometric distribution. This can be generalized to larger tables, and I believe at least some of the R packages estimate this p-value using a Monte Carlo method. An additional issue to consider is that these p-values may actually be conservative. The "mid-p" value can be used to correct for this, but I am not certain about the theoretical underpinnings of this approach.

Finally, I am not familiar with the exact2x2 package, but if you believe the p-value produced by this package is reasonable, it doesn't appear that you have issues with power. Saying a test is not powerful means that we are concerned that we will not correctly reject the null hypothesis when it is false. Given that the test the exact2x2 package conducted resulted in rejection of the null hypothesis for common significance levels I would think that the lack of power is less of a concern.

$\endgroup$
1
  • $\begingroup$ Thanks for the answer, since some of my cells are below 5, i was about to do a fisher test, but then i came to know about Barnard test and exact package, in its tutorial, i found this "Unconditional tests (such as Barnard’s and Boschloo’s exact tests) are more powerful alternatives than conditional tests (such as Fisher’s exact test)." that was the reason which made me think that maybe using Barnard test might be a better idea. but as you are saying a Fisher test would suffice. $\endgroup$ – user3015703 Aug 17 '14 at 4:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.