Groups we seek to compare (e.g., a treatment group and control group) may differ in ways that constrain our ability to do so. Often, potential outcomes may differ systematically across groups, such as in the case of observational studies and a broken experiment.

I've heard this general type of problem discussed in terms of selection or selection bias, meaning our estimate of the effect of some treatment is inaccurate due to differences in potential outcomes across groups. I've also heard of composition effects (and less frequently composition bias), where the covariates and potential outcomes of units vary across groups. I don't find any references that directly compare and contrast the two terms.

What are the differences in meaning (or usage) between the two terms selection bias and composition effect? It seems to me that selection bias describes an obstacle to unbiased estimation and composition effect/bias describes an (often undesirable) characteristic of a dataset. Is this correct? Is there more beyond that?


1 Answer 1


I think both terms in the context you give refer to the same bias in assignment mechanism of groups in potential outcome framework.

For example, if you use observational dataset to analyze effectiveness of aspirin in treating headaches:

  • people who decide to take aspirin suffer overall more severe headaches and perhaps chronical, and still feel the headaches after taking the treatment
  • people who do not take aspirin may recover since their headaches are light

Since people make their own decision to take treatment, this assignment mechanism creates difference in covariates among the two groups, and you may have a significant bias on the average treatment effect estimate from this observational dataset.

You can call this bias a selection bias as only severe patients are selected for treatment in the natural setting. You can also call this a composition bias since the distribution of covariates of the treatment group significantly differ from that of the control group. But it all boils down to bias in assignment mechanism of the groups.

Interesting reading: https://en.wikipedia.org/wiki/Rubin_causal_model#The_perfect_doctor


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.