Simpson's Paradox vs. Berkson's paradox

Question

Can someone explain what is the difference between the two? They seem to me to be identical. In both paradoxes you start from a narrow distribution and find out the correlation switches when you move to the full distribution. So what's the difference actually?

Current Answers Review:

• Mickybo Yakari points that Berkson's Paradox relates to the (potentially wrongful) sampling of the data. While Simpson's paradox doesn't relate to sampling risks - but rather the analysis of the data (conditioning it on some variable or not).

• Acccumulation makes the same distinction between selection bias (Berkson's) and categorization bias (Simpson's), and claims that Berkson's can be viewed as a subset of Simpson's.

• Noah introduces the notion of underlying "truth". In Simpson's the conditioning (or categorization) on a confounder variable reveals the truth, and not doing so is confounding; while in Berkson's the conditioning (or sampling) on a colider variable, hides the truth.

Answer 1

Both Simpson's paradox and Berkson's paradox are statistical phenomena in which a surprising disparity is observed but they differ with respect to the reason they arise. Let's describe them in a few words and determine how they differ.

Simpson's paradox is a statistical phenomenon in which a trend between two variables occurs in several different groups of data, formed according to the values taken on by a conditioning variable, but disappears or reverses when the groups are combined. The disparity lies between the disaggregation-based conclusion and the aggregation-based conclusion and is not caused by a lack of data within any partition subset of the data but rather by the relative sizes of the partition subsets (a matter of calculus of proportions).

Berkson's paradox arises from the fact that the sample is collected in such a way that some individuals of the population (characterised by a conditioning variable) are less likely to be selected than others.

In

Pearl, J. (2013), Linear Models: A Useful "Microscope" for Causal Analysis, Journal of Causal Inference, 1.1, 155-170,

using the language of graphical models, the author explains:

Selection bias is symptomatic of a general phenomenon associated with conditioning on collider nodes[…] The phenomenon involves spurious associations induced between two causes upon observing their common effect, since any information refuting one cause should make the other more probable. It has been known as Berkson Paradox (Berkson, 1946), "explaining away" (Kim and Pearl, 1983) or simply "collider bias".

This is problematic because it may then turn out that, due to the conditioning variable and the entailed biased sampling, the sample accurately represents a certain subset of the population but not the whole population.

Here is an additional paper to further one's understanding of both paradoxes:

Pearl, J. (2014), Understanding Simpson's Paradox, The American Statistician, 1.68, 8-13.

The author notes that Simpson himself noticed that, depending on the story behind the data, the more sensible (Simpson's words), is sometimes compatible with the disaggregated analysis and sometimes with the aggregated analysis. He provides Simpson's classical examples for both.

Answer 2

Simpson's paradox occurs when conditioning on a variable reveals the true association and failing to condition on a variable reveals a false association. Berkson's paradox occurs when (inadvertently) conditioning on a variable reveals a false association, and the true association would have been revealed had no conditioning occurred.

Simpson's paradox is caused by confounding, i.e., the presence of a common cause of both variables. Failing to condition on the confounder yields a biased association between the variables, but conditioning on the confounder corrects this bias. Berkson's paradox is caused by selection, i.e., conditioning on a common consequence both variables, which is called a collider. Conditioning on a collider induces an association between the causes of the collider.

Answer 3

Simpson's is a subset of Berkson's. Suppose the overall data has positive correlation, and is split into Set A and Set B. If Set A has a negative correlation, then that's Berkson's paradox. If both Set A and Set B have negative correlation, that's Simpson's.

Also, with Simpson's, both Set A and Set B have to be collected (otherwise you don't know both have a negative correlation). With Berkson's, only Set A has to be collected. So generally, misleading correlations that result from categorization are treated as Simpson's paradox, while those that result from selection bias are treated as Berkson's.

Answer 4

The difference between the two paradoxes becomes clear when we look at some causal DAGs.

Simpson's paradox

Simpson's paradox is a puzzle about confounding [0]. A recent example of this paradox can be seen in COVID-19 data on mortality by race as described in Judea Pearl's blog. The paradox: non-hispanic whites have a higher mortality if we look at the aggregated data. But: Disaggregated by age whites have a lower mortality in every age group. The corresponding DAG: Why is this happening? We want to figure out the relationship A (Race ⇒ Death from COVID). But as we can see from the graph, Age is a confounder for this relationship and therefore looking at the observational data Pr(Death|Race) will not give the right answer. Once we look at Pr(Death|Race, Age) the effect of the variable Race reverses. Now why this can happen may seem unintuitive, yet as Pearl explains in Understanding Simpon's Paradox:

(a+b)/(c+d) > 1 does NOT imply a/c and b/d > 1.

And which you should be looking at (the aggregated or stratified data) depends on the CDAG and the research question, explained at length in the Book of Why.

Berkson's Paradox

Berkson's paradox arises by conditioning on a collider. This can appear paradoxical if it happens as an artifact of the study design, for example during participant selection.

Say you're investigating whether there is any relationship between catching COVID and having any other illnesses. Further assume that in the general population COVID is independent of Other illnesses. Now if you base your study on hospital patients only you'll find Pr(COVID|not Other illnesses) = 1! After all, they must have been hospitalized for something. Conditioning on hospitalization has made the two variables dependent where they previously weren't. More specifically, conditioning on the collider has unblocked the causal path between the two variables.

So: Simpson's paradox is confounder bias, Berkson's paradox is collider bias. In both cases looking at the DAG brings clarity.

[0] Book of Why p.202