How to combine p values? [closed]

Question

I have three datasets A, B and C. A and C can construct a contingency table, and I can get a p value (pValue1) with Fisher's exact test. B and C can construct another contingency table, and I can get another p value (pValue2) with Fisher's exact test.

I want to know how to combine these two p values and get a single one. One possible method is using Fisher's method, but it tends to generate a new value close to the minimum one. I want to use some other method to tend to generate one close to the maximum one, namely both reject the null hypothesis.

Can anyone tell me which method I can use?

Answer 1

Since it is evidently meaningful to form a contingency table from A and C, and from B and C, I suspect what you call "datasets" might be more commonly referred to as variables within a single dataset. Moreover, these seem to be categorical variables. I gather you want to test if there is an association between C and either A or B. You can determine that within a single model—that is, Fisher's exact test is inappropriate here and you don't need to combine p values. Instead, the appropriate analysis is a log-linear model.

Here is a quick R demonstration:

d = data.frame(A=    rep(c("yes","no"), each=40),
               B=rep(rep(c("yes","no"), each=20), 2),
               C=  c(rep(c("yes","no"), times=c( 5,15)),
                     rep(c("yes","no"), times=c(10,10)),
                     rep(c("yes","no"), times=c(10,10)),
                     rep(c("yes","no"), times=c(15, 5)) ) )
tab = table(d)
margin.table(tab, margin=c(1,3))
#      C
# A     no yes
#   no  15  25
#   yes 25  15
margin.table(tab, margin=c(2,3))
#      C
# B     no yes
#   no  15  25
#   yes 25  15
margin.table(tab, margin=c(1,2))
#      B
# A     no yes
#   no  20  20
#   yes 20  20

library(MASS)  # we use this package to get the loglm() function
loglm(~C + A*B, tab)
# Call:
# loglm(formula = ~C + A * B, data = tab)
# 
# Statistics:
#                       X^2 df   P(> X^2)
# Likelihood Ratio 10.46496  3 0.01500047
# Pearson          10.00000  3 0.01856614

I have a more extensive discussion here: $\chi^2$ of multidimensional data.

Answer 2

It seems you want to simultaneously test AvBvC in a 2x3 contingency table? It's not entirely clear what you are asking for.

You can calculate $\chi^2$ tests or Fisher Exact Tests (troublesome by hand) on tables larger than 2x2. For $\chi^2$ , the procedure is the same as in a 2x2 table - calculate the expected frequency of each cell as $$ E_{ij} = (RowTotal_i*ColumnTotal_j)/Total $$ Your $\chi^2$ statistic is then $$ \sum_{ij} {(O_{ij}-E_{ij})^2 \over E_{ij} }$$ with degrees of freedom $ (i-1)(j-1) $

Caveat here - your question makes it sound like you are fishing for a null rejection by throwing a battery of tests at your dataset. Recall that we reject the null if the p-value we obtain from a test is below an arbitrary point. By throwing multiple tests at a dataset, you increase by pure chance that you one of these tests will pass that arbitrary point. Thus this is generally frowned upon without correction for those multiple tests.