9
$\begingroup$

I have two groups (A and B) that each produce ORs of 1.44 and 1.50. However, if I combine the frequencies for the two groups to create a pooled dataset, I get an OR of 1.40.

I get that it's not going to be a nice simple weighted mean or anything, but I would have expected that the pooled OR would fall within the range of the two OR's at a minimum. After sever days of puzzling over trying to pin down the cause, I've given up and am turning to the collective brain power here.

Here's my data:

Group A frequencies

Control Test
Low 1374 1422
High 4759 7062

Produces an OR of 1.44

Group B frequencies

Control Test
Low 825 534
High 4033 3914

Produces an OR of 1.50

Pooled frequencies

Control Test
Low 2199 1956
High 8792 10976

produces an OR of 1.40

Any thoughts on what is going on here? I'm using these two groups to show the pattern, but my data is actually split amongst many more groups.

I'm referring to it as a paradox because if I include all of my groups, despite the fact that nearly all of the groups produce OR's in the 1.2-1.8 range (a couple produce OR's in the .95-1.0 range), the pooled version becomes basically null (OR = 1.00). It feels like there's a Simpson's paradox of sorts.

EDIT: Per @COOLSerdash below, this paper explains the "noncollapsibility" of ORs. After playing around a little bit, it appears that if the larger groups have RRs that are very different from the rest of the groups, it will have a strong "distorting" effect.

$\endgroup$
9
  • 4
    $\begingroup$ This does look like a version of Simpson's paradox (not really the "plain vanilla" version, which is driven by strongly unequal sizes of the groups, whereas your groups are quite similar in size). That said, what is your question? You seem to have calculated everything correctly, so it seems like ORs in subgroups simply do not behave the way we would assume. What else are you looking for? $\endgroup$ Apr 18 at 13:12
  • 2
    $\begingroup$ I think this relates to the non-collapsibility of the odds ratio. See table 1 in this paper. $\endgroup$ Apr 18 at 13:17
  • 4
    $\begingroup$ Relative risks are collapsible, odds ratios are not. This paper discusses this among other things. $\endgroup$ Apr 18 at 13:47
  • 2
    $\begingroup$ @COOLSerdash yep! That seems to be it. If you post it as an answer, I'll mark the question answered. $\endgroup$
    – S Robidoux
    Apr 18 at 13:49
  • 2
    $\begingroup$ As a general mathematical/statistical rule: taking averages of ratios doesn't make sense (and this includes averages of averages). This is the same principle underlying Simpson's paradox. If we're looking for an intuition-improving idea, I nominate this one :) $\endgroup$ Apr 19 at 17:55

3 Answers 3

18
$\begingroup$

You have discovered the property of non-collapsibility of the odds ratio. Very briefly, this means (among other things) that the pooled odds ratio is not a weighted average of the group-specific odds ratios. The relativ risk on the other hand is collapsible. The papers below go into much more detail.

References

Abdulmajeed, J., Kostoulas, P., Shi, Z., & Doi, S. A. (2023). Noncollapsibility of the odds ratio unraveled. Current Opinion in Epidemiology and Public Health, 2(3), 32-38.

Burgess, S. (2017). Estimating and contextualizing the attenuation of odds ratios due to non collapsibility. Communications in Statistics-Theory and Methods, 46(2), 786-804.

Cummings, P. (2009). The relative merits of risk ratios and odds ratios. Archives of pediatrics & adolescent medicine, 163(5), 438-445.

Greenland, S. (2021). Noncollapsibility, confounding, and sparse-data bias. Part 1: The oddities of odds. Journal of clinical epidemiology, 138, 178-181.

Greenland, S. (2021). Noncollapsibility, confounding, and sparse-data bias. Part 2: What should researchers make of persistent controversies about the odds ratio?. Journal of clinical epidemiology, 139, 264-268.

Huitfeldt, A., Stensrud, M. J., & Suzuki, E. (2019). On the collapsibility of measures of effect in the counterfactual causal framework. Emerging themes in epidemiology, 16, 1-5.

Schuster, N. A., Twisk, J. W., Ter Riet, G., Heymans, M. W., & Rijnhart, J. J. (2021). Noncollapsibility and its role in quantifying confounding bias in logistic regression. BMC medical research methodology, 21, 1-9.

$\endgroup$
8
$\begingroup$

The solution to the dilemma is to quit failing to condition on things you should condition on. If there is a factor affecting the outcome, it should be conditioned on. Simpson's "paradox" is nothing more than incomplete conditioning. Crude marginal unconditional estimates have very limited utility when there is outcome heterogeneity. And the fact that odds ratios are not collapsible is not a negative.

$\endgroup$
2
  • 5
    $\begingroup$ For future reference, Harrell provides an argument for the claim he makes in the last sentence of this answer in the 3rd paragraph of the background section of one of his blog posts here: Unadjusted Odds Ratios are Conditional. $\endgroup$ Apr 19 at 19:19
  • 1
    $\begingroup$ 100% agree, and this is the conclusion we arrived at - the pooled OR isn't informative. $\endgroup$
    – S Robidoux
    Apr 26 at 16:24
0
$\begingroup$

It seems that some confounder is affecting groups A and B differently, while they also have different proportions of control and test groups. This causes Simpson's paradox in the pooled frequencies:

Group A has a larger proportion of test participants, and overall relatively low odds in both control and test group. Group B has a smaller proportion of test participants, and overall higher odds in both groups.

This means that in the pooled dataset, being a test participant correlates with being in group A, which correlates with a low outcome (lower odds).

$\endgroup$
1
  • $\begingroup$ It's not a Simpson's paradox in the sense that a Simpson's paradox can be explained by weightings. The non-collapsibility of OR's can't be dealt with this way. See the various answers/comments above this one. $\endgroup$
    – S Robidoux
    Apr 26 at 16:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.