14
$\begingroup$

Recently, I have been reading about Propensity Score Matching :

enter image description here

If I have understood this correctly, Propensity Score Matching is used to construct control/treatment groups in scientific studies, in such a way that individuals in the control group are as similar as possible to individuals in the treatment group. In other words, an individual in one of these groups is "matched" to an equivalent individual in the other group. This is done to reduce the risk of "latent and unobserved" variables cofounding the effects of the treatments - and that "apples are compared to apples" instead of "apples to oranges".

At the surface, this seems to be very important - after all, if we are testing the effects of some pharmaceutical drug on two similar groups of people, we would like to avoid the risk of one these groups being comprised primarily of Olympian athletes and the other of senior citizens (assuming the goal of the study is to compare the effects of the drug on similar groups of people).

My Question: Just to clarify - do most researchers attempt to implement some form of Propensity Score Matching when conducting these kinds of statistical studies? Is this a "must"?

If some form of Propensity Score Matching is not implemented properly relative to the objective of the study, does this pose a high risk of invalidating the statistical study? According to the Wikipedia article (https://en.wikipedia.org/wiki/Propensity_score_matching), Propensity Score Matching was popularized in the 1980's - does this suggest that statistical studies conducted prior to the 1980's were more likely to suffer from these kinds of undesired variable confounding effects?

$\endgroup$
5
  • 2
    $\begingroup$ The alternative is to hope that randomizing subjects into two (or three or more) groups works as intended. If 'fake twins' (or 'triplets' or blocks) can really be found, one might reduce variability and might be able to use a paired test instead of a two-sample test. In practice, I'd say 'score matching' is always desirable and too seldom achievable. $\endgroup$
    – BruceET
    Jan 7, 2022 at 4:07
  • 6
    $\begingroup$ No, propensity scoring is not a "must" for causal inference. Propensity scores can be useful, but they come with drawbacks. See Hernán & Robins' critique in Chapter 15: Propensity models, structural models, predictive models of Causal Inference: What If? $\endgroup$
    – Alexis
    Jan 7, 2022 at 4:09
  • 1
    $\begingroup$ Agreed with both comments above (+1). I would also add, that it could be reasonably argued that when dealing with observational data, then it is "strongly desired" to clarify why propensity scores (or any other quasi-experimental methodology) are not used i.e. why we would consider a "convenience" sample as good as one that would be generated under experimental design principles. $\endgroup$
    – usεr11852
    Jan 7, 2022 at 4:29
  • $\begingroup$ If it is, most science is an utter failure. (Nearly true, any way.) $\endgroup$
    – Nick Cox
    Jan 7, 2022 at 15:13
  • $\begingroup$ Relevant discussion here stats.stackexchange.com/questions/481110/… $\endgroup$ Jan 8, 2022 at 2:50

2 Answers 2

26
$\begingroup$

Propensity score methods are one type of method used to adjust for confounding. There are several other methods that rely on different assumptions. Some of the most popular include difference-in-differences, which relies on an assumption about stability over time, and instrumental variable analysis, which relies on an assumption about randomization of some other variable. A third class of methods includes methods that rely on an assumption that all confounding variables have been measured. I highly recommend this 2020 article by Matthay et al. for a comparison of these methods.

Propensity score methods fall in the latter class. Other methods also fall in this class, including regression adjustment, "g"-methods, and doubly-robust methods. These are all different ways of adjusting for confounding by measured covariates by conditioning on them in certain ways. They differ primarily in their statistical performance under various assumptions about the functional form of the treatment and outcome processes.

There are several ways to use propensity scores, including matching (which you described), weighting, subclassification, and regression adjustment, and there are ways to perform each of these methods without propensity scores. I mention all of this so that you see propensity scores as one particular implementation of methods that themselves are members of a broad class of methods that is one of several classes of methods one can use to adjust for confounding. Propensity score methods are not necessarily superior to any of them, and their ubiquity is likely a cultural artifact rather than truly justified by their statistical performance.

Here are a few reasons (and rebuttals) for why propensity score may be popular:

  • They are easy to implement (but only in their most basic, poorest performing way; to use them well requires extensive knowledge)
  • They are easy to explain to lay audiences (but so are many methods that don't involve propensity scores, like other matching methods)
  • They tend to be effective at removing bias due to confounding (but several methods are demonstrably better, especially better than propensity score methods as most commonly used)
  • They separate the design and analysis phase, leading to more replicable research and decreasing model dependence (but when used poorly can increase model dependence and are not immune to snooping and nefarious or misguided use)
  • They are implemented in most statistical software (but so are many other methods, and they are implemented differently in each software)
  • They are a form of dimension reduction in high-dimensional datasets (but there are other ways to reduce dimensionality, and still propensity scores are used even to adjust for a few covariates)
  • They rely less on modeling assumptions than regression-based methods (but there are many other methods that also allow for extreme flexibility with often improved performance)
  • They sound fancy and make the analyst look sophisticated (but experienced statisticians can easily point out the errors amateur users constantly make)

(You might think I am biased against propensity scores, but check the tag and see my involvement. I'm also the author of several R packages to facilitate the use of propensity score methods.)

In my opinion, propensity scores are overused (or, at best, under-justified) in the medical literature. There are so many better performing and more sophisticated methods that rely on the same assumptions as propensity score methods do that are under-appreciated in medical research, often because the analysts and reviewers in medical research are not familiar with them. I hope to encourage people to consider propensity scores as one option in a vast sea of options, each of which has its own advantages and disadvantages that make it more or less suitable for a given problem. To decide which option is the best for a given problem requires the assistance of a statistician specially trained in the area of causal effect estimation.

$\endgroup$
5
  • $\begingroup$ Very informative answer. It will be very useful if you would mention some important alternatives available in "a vast sea of options". $\endgroup$
    – rnso
    Jan 7, 2022 at 17:11
  • 1
    $\begingroup$ (+1 obviously) @rnso: "difference-in-differences", "instrumental variable analysis", "regression adjustment, 'g'-methods, and doubly-robust methods" are mentioned in the text. $\endgroup$
    – usεr11852
    Jan 8, 2022 at 1:14
  • 1
    $\begingroup$ 1/2 +1 (fo' realsies. :) I think it is worthwhile pointing out some of the difficulties propensity scores present to communication of and transportability of substantive findings. From Hernán & Robins: Effect modification across propensity strata may be interpreted as evidence that decision makers know what they are doing, e.g. that doctors tend to treat patients who are more likely to benefit from treatment (Kurth et al 2006). However, the presence of effect modification by $\pi(L)$ may complicate the interpretation of the estimates. $\endgroup$
    – Alexis
    Jan 8, 2022 at 4:28
  • 2
    $\begingroup$ 2/2 Consider a situation with qualitative effect modification: “Doctor, according to our study, this drug is beneficial for patients who have a propensity score between 0.11 and 0.93 when they arrive at your office, but it may kill those with propensity scores below 0.11,” or “Ms. Minister, let’s apply this educational intervention to children with propensity scores below 0.57 only.” The above statements are of little policy relevance because, as discussed in the main text, they are not expressed in terms of the measured variables $L$. $\endgroup$
    – Alexis
    Jan 8, 2022 at 4:29
  • 4
    $\begingroup$ @Alexis true. The propensity score is balancing score, a tool for achieving balance, but IMO it has no other interpretation (i.e., it doesn't actually represent the propensity to receive treatment). I do think statements like "Those least likely to attend college are those most likely to benefit from it" are helpful and meaningful (which is an example of interpreting the propensity score as an effect modifier), but understanding effect modification as a function of covariates is much more productive. $\endgroup$
    – Noah
    Jan 8, 2022 at 4:34
11
$\begingroup$

As Alexis pointed out, propensity score matching (PSM) is one of many tools we have in causal inference. Another one is Inverse Probability Weighted Estimator (IPWE). You can also use causal discovery to infer a causal diagram and use do-calculus to estimate the causal effect. Or make use of instrumental variables estimation. I'm just throwing a lot of names here (though with links, in case you want to see more about it) with a single intent: To show you there are many tools you have when your goal is causal inference in observational data. They all do something specific and have their advantages and limitations.

Should we always use such tools when we want to do causal inference in observational data? Yes. Are all scientific studies about this? No. What's the best one? It depends on what you want, how you want and what you have :-).

One last thing: Latent and unobserved are synonymous in this context. Latent confounders and unobserved confounders refer to the same thing. Besides, confounding is not your only enemy when it comes to inferring causality. Collider bias is another one and propensity score matching does not account for bias due to censoring.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.