Definition of selection bias vs confounding bias

Question

I've been learning about causal inference, having read Pearl's Primer and Parts I and II of "What If?". I was under the impression that the definition of "There is confounding" was

(T,Y) are are said to be confounded if $$\exists (t,y) s.t. \mathbb{P}(Y^t=y) \neq \mathbb{P}(Y=y|t).$$

I was further of the impression that confounders are, loosely speaking, the variables we must adjust for. This meant that while confounding was clearly defined, in general a confounder was not, because there are multiple valid sets of variables one can adjust for (or that the definition is at the very least kind of complicated, something to the tune of: a variable is a confounder if together with other variables it can adjust for the bias (which to me means allow us to calculate the causal effect)). I think there was a paper by Tyler VanderWeele on this, but that's beside the point. To me, all bias was thus confounding bias.

I've often been encountering ressources that differentiate between selection bias and confounding bias, and it confuses me. To me, the SCM framework (ie, explanation in Pearl's primer) seems like it encompasses all scenarios. They also seem to say that the two must be treated differently.

In simple cases (e.g., when there are three variables and one is a common cause/common effect), it seems clear that confounding bias happens when we don't condition on the common cause and selection bias happens when we condition on the collider (common effect). In general, I'm not sure. Can we see selection bias as any association path that flows through a collider?

What is a formal definition of selection bias (and do I perhaps misunderstand what confounding bias is)? How should they be treated differently?

Thanks very much for clarifying this issue.

Answer 1

You're correct that terms like confounding bias and selection bias are often loosely or implicitly defined, as much of the focus tends to be on prognosis and identification paths in cases where these biases are isolated. However, in the Structural Causal Model (SCM) framework, by Dr. Judea Pearl, both types of bias can be diagnosed directly from the causal graph.

Confounding bias is easily detected graphically when no other biases are present. Let's consider the target query to be the interventional distribution after a binary point-exposure $A$: $p(y \mid \text{do}(A = a))$, with $a \in \{0, 1\}$, and the causal graph:

$$ \mathcal{G}_1: W\rightarrow A\rightarrow Y\leftarrow W $$

Now, consider the statistical estimand:

$$ q_Z(y\mid a) := \int p(y\mid A=a,Z)\, \text{d}P(Z) $$

According to the rules of do-calculus, $p(y \mid \text{do}(A = a))$ is identified by $q_Z(y \mid a)$ in $\mathcal{G}_1$ if: (i) $Z$ blocks all non-causal/backdoor paths from $A$ to $Y$, and (ii) $Z$ contains no descendants of $A$. These are the classical criteria for backdoor admissibility as defined by Dr. Pearl.

If we try $Z = \varnothing$, then $q_\varnothing(y \mid a) = p(y \mid A = a)$ does not identify the interventional distribution because $Z = \varnothing$ does not block the backdoor path $A \leftarrow W \rightarrow Y$. This means the statistical estimand $p(y \mid A = a)$ is affected by confounding bias, and, as a consequence (not as a premise):

$$ p(y\mid\text{do}(A=a)) \neq p(y\mid A=a), $$

where the inequality holds in distribution. In this case, confounding is the only source of bias.

Selection bias, on the other hand, is traditionally classified into two cases: coming from preferential selection of units into the data, or coming from inadmissible stratification. Recently, some researchers argue that the latter should be viewed as a form of confounding rather than true selection bias. For instance, when conditioning and marginalizing on a collider (e.g., $A \rightarrow C \leftarrow Y$), since this is a violation of backdoor admissibility. Selection bias should require some form of selection instead!

Selection bias, specifically from preferential selection, can also be diagnosed graphically by introducing selection nodes (e.g., missingness indicators). Consider a second causal graph where $R_Y \in \{0, 1\}$ indicates whether an observation of $Y$ is in fact selected ($R_Y = 1$) or missing ($R_Y = 0$) in the data:

$$ \mathcal{G}_2: A\rightarrow Y\leftarrow U\rightarrow R_Y $$

If there were no missingness, $q_\varnothing(y \mid a) = p(y \mid A = a)$ would identify the interventional distribution, as there are no backdoor paths between $A$ and $Y$. However, selection bias occurs here because we end up recovering $p(y \mid \text{do}(A = a), R_Y = 1)$, which refers to a stratum subpopulation that might differ from the target population. This arises because we can only use the selected observations of $Y$, which might have characteristics that differ from the entire population.

To recover the target query in $\mathcal{G}_2$, adjusting for $U$ solves the problem, with $q_U(y \mid a, 1)$:

$$ q_U(y\mid a,1) := \int p(y\mid A=a,U,R_Y=1)\, \text{d}P(U), $$

Here, adjusting for $U$ blocks the statistical dependence between the outcome and the selection mechanism without introducing confounding bias as $U$ is backdoor admissible itself.

Finally, consider a more complex graph:

$$ \mathcal{G}_3: W\rightarrow A\rightarrow Y\leftarrow W\quad \sqcup\quad Y\leftarrow U\rightarrow R_Y $$

In this case, $q_\varnothing(y \mid a, 1) = p(y \mid A = a, R_Y = 1)$ is affected by both confounding and selection biases, but adjusting for both $W$ and $U$ resolves it. This is because $\{W,U\}$ is backdoor admissible and $d$-separates $Y$ from $R_Y$. Hence, $q_{\{W, U\}}(y \mid a, 1)=p(y\mid\text{do}(A=a))$.

There are more intricate graphical scenarios, especially those involving latent variables and multiple selection or missingness mechanisms, where it becomes impossible to address both confounding bias and selection bias at the same time.

I leave some interesting references here:

• ^{Sander Greenland. Quantifying biases in causal models: classical confounding vs collider stratification bias. Epidemiology, 14(3):300–306, 2003}

• ^{Tyler J. VanderWeele and Ilya Shpitser. On the definition of a confounder. Annals of Statistics, 41(1):196–220, 2013}

• ^{Miguel A. Hernán, Sonia Hernández-Díaz, and James Robins. A structural approach to selection bias. Epidemiology (Cambridge, Mass.), 15(5):615 625, 2004. 10.1097/01.ede.0000135174.63482.43}

• ^{Haidong Lu, Chanelle J Howe, Paul N Zivich, Gregg S Gonsalves, and Daniel Westreich. The Evolution of Selection Bias in the Recent Epidemiologic Literature—A Selective Overview. American Journal of Epidemiology, page kwae282, 08 2024. ISSN 0002-9262. 10.1093/aje/kwae282. URL https://doi.org/10.1093/aje/kwae282}

• ^{Haidong Lu, Gregg S Gonsalves, and Daniel Westreich. Selection Bias Requires Selection: The Case of Collider Stratification Bias. American Journal of Epidemiology, 193(3):407–409, 11 2023. ISSN 0002-9262. 10.1093/aje/kwad213. URL https://doi.org/10.1093/aje/kwad213}

• ^{Daniel Westreich. Berkson’s bias, selection bias, and missing data. Epidemiology, 23(1):159–164, 2012. 10.1097/EDE.0b013e31823b6296. PMID: 22081062; PMCID: PMC3237868}

• ^{Miguel A. Hernán. Invited Commentary: Selection Bias Without Colliders. American Journal of Epidemiology, 185 (11):1048–1050, 05 2017. ISSN 0002-9262. 10.1093/aje/kwx077. URL https://doi.org/10.1093/aje/kwx077}

• ^{Maya B Mathur, Ilya Shpitser, and Tyler VanderWeele. A common-cause principle for eliminating selection bias in causal estimands through covariate adjustment. OSF Preprints ths4e, Center for Open Science, January 2023. URL https://ideas.repec.org/p/osf/osfxxx/ths4e.html}

• ^{Elias Bareinboim, Jin Tian, and Judea Pearl. Recovering from selection bias in causal and statistical inference. Proceedings of the AAAI Conference on Artificial Intelligence, 28(1), Jun. 2014. 10.1609/aaai.v28i1.9074. URL https://ojs.aaai.org/index.php/AAAI/article/view/9074}

Answer 2

Adding some brief (and hopefully intuitive) thoughts to Johan's brilliant response:

Confounding primarily stems from non-random treatment assignment/exposure receipt, and selection bias stems from non-random sample selection/attrition. Selection bias results from a change in the sample being studied or analyzed, while confounding does not.