5
$\begingroup$

How might one test the null hypothesis that two 2x2 tables (of frequencies) were drawn from the same population?

More specifically, I have two 2x2 samples of unequal sample size. I would like to test the hypothesis that these 2 samples came from the same population. The sample sizes are not of interest to me.

The tables below illustrate my data rather closely. The two samples consist of cases that showed up at two facilities in a one year period. I presume the two 2x2 tables are unbiased samples of the geographic areas of the two facilities. I do not anticipate receiving more data in the near future.

I am only interested in comparing the survival rates across the two samples.

Update
Here is my attempt to state the null and alternative hypotheses for comparing two 2x2 tables of this type.

In general, the research question of whether these two samples are comparable (i.e., come from the same underlying population) seems to require three null hypotheses, each stated in terms of equality. The alternative hypotheses replaced quality with inequality:

  1. Diseases A and B occur with the same proportion in populations A and B (the populations from which the samples were drawn).
  2. The death rates for both diseases are the same in populations A and B.
  3. The interaction between disease and death rate is the same in populations A and B.

In this data set, the null and alternative hypotheses seem to omit #1 and #3, since the proportion of diseases A and B are identical in the two samples, so there is no indication of a difference in disease rate.

data_two_2x2_samples

$\endgroup$
13
  • 1
    $\begingroup$ You really need to add some context here, sample size, and most important, how where the sampling done. Assuming it was simple random sampling, you want to test both row and col probabilities, and maybe the odds ratio. With simple random samplingthat leads to a test of two multinomial distributions are the same. $\endgroup$ Commented May 5 at 17:19
  • 1
    $\begingroup$ "I would like to test the hypothesis that these 2 samples came from the same population. ". Why do you want to know that, ultimately? For example, are you interested in knowing if the risk of dying of a specific disease is higher in one of the two areas? Maybe something else? $\endgroup$
    – J-J-J
    Commented May 6 at 9:21
  • 3
    $\begingroup$ You have a $2\times2\times2$ contingency table and appear to want to address the hypothesis that the expectations of cells in one "pane" equal those in the other. That's three linearly independent contrasts that you can test using the standard machinery in a GLM. The specifics of what those contrasts are and which GLM you formulate depend on the details of how you collected these data and how you wish to model the potential variation, which is why others have been focused on those issues. $\endgroup$
    – whuber
    Commented May 6 at 13:36
  • 1
    $\begingroup$ @J-J-J I am only interested in comparing the survival rates across the two samples. $\endgroup$
    – Joel W.
    Commented May 6 at 14:30
  • 1
    $\begingroup$ @whuber. I will add null and alternative hypotheses to the question. Pls let me know if they make sense. $\endgroup$
    – Joel W.
    Commented May 8 at 13:24

1 Answer 1

6
$\begingroup$

It is still not entirely clear what is your research question. You ask if the tables come from the same population, which would imply a model with the same marginal frequencies, and the same odds ratio. If that really is the question, and we assume simple random sampling (this seems consistent with what you say), then this is a question if two multinomial distributions are the same. Or with other words, if all the populations proportions are equal. This can be done with (in R)

prop.test(matrix(c(1, 1, 7, 7, 3, 4, 21, 28), ncol=2),) 

    4-sample test for equality of proportions without continuity correction

data:  matrix(c(1, 1, 7, 7, 3, 4, 21, 28), ncol = 2)
X-squared = 0.25714, df = 3, p-value = 0.9679
alternative hypothesis: two.sided
sample estimates:
prop 1 prop 2 prop 3 prop 4 
  0.25   0.20   0.25   0.20 

Warning message:
In prop.test(matrix(c(1, 1, 7, 7, 3, 4, 21, 28), ncol = 2), ) :
  Chi-squared approximation may be incorrect

The result is not surprising given the small samples. This test is formally equivalent to the test of independence in the contingency table

tab <- matrix(c(1,1,7,7,3,4,21,28), ncol=2)
> tab 
     [,1] [,2]
[1,]    1    3
[2,]    1    4
[3,]    7   21
[4,]    7   28

and to get a more precise p-value we can simulate that test

chisq.test(tab, simulate.p.value=TRUE, B=1000000)

    Pearson's Chi-squared test with simulated p-value (based on 1e+06 replicates)

data:  tab
X-squared = 0.25714, df = NA, p-value = 0.9568

Note that with the very small sample sizes any test will have low power!

But, maybe in reality you should ask if the association (for example measured by the odds ratio) are equal? At least, with small sample size it is better to ask a more focused question of the data, to have better chance (that is, power) of detecting a difference, if it exists.

That leads to a test if odds ratios in two tables are equal, which can be done with logistic regression. But sample size is really to small for that to make much sense, so I will not show that. Maybe better to simply show sample estimates of odds ratios, with standard errors.

$\endgroup$
3
  • $\begingroup$ As to my research question, I am only interested in comparing the survival rates across the two samples. $\endgroup$
    – Joel W.
    Commented May 6 at 14:28
  • 1
    $\begingroup$ You should really add that as an edit to the original post, as an edit. But, the sample sizes are to small to get much. A confidence interval will be very wide. $\endgroup$ Commented May 6 at 15:08
  • $\begingroup$ Yes the confidence interval will be wide, but I want to learn the method so I can apply it to other samples as needed. (I edited the question.) $\endgroup$
    – Joel W.
    Commented May 6 at 15:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.