9
$\begingroup$

I have asked here whether, in order to establish causal relationships, the treated group and the control group must be similar on all covariates.

The answer was no, if we control for the covariates in an OLS regression.

So what are the use cases for Propensity Score Matching?

Why I can't just run an OLS regression controlling for the covariates on which the two groups (treated and control) differ?

EDIT: The problem is that my colleague is saying that the OLS I proposed in the linked question will not work and we need to use Propensity Score Matching. Wikipedia is saying the opposite:

This blocking can be done either by adding the confounding variable as a control in regression, or by matching on the confounding variable.

So, according to Wikipedia, either adding the variables as controls or using Propensity Score Matching will work just fine both ways

EDIT: The topic is also covered in this video at minute 12:00

$\endgroup$
1
  • 3
    $\begingroup$ Besides the answers you've received: You are not necessarily fine in any of the cases in a non-randomized study. Remember that you may very well have balanced/adjusted/whatever for all the observed covariates, but important unobserved confounding may still be present. That's of course particularly an issue, if you collected only a small amount of covariate information and/or have such a small sample size that it's tough to account for all the information you have. $\endgroup$
    – Björn
    Apr 30 at 12:23
11
$\begingroup$

You need to distinguish between uses of propensity scores for matching of cases versus for more general adjustments.

The discussion on this page suggests that there isn't much of a use case for propensity score matching. Among other problems, there is seldom much to be gained by throwing away information. Yet that is what matching cases does, with additional problems introduced by using propensity scores for the matching.

That said, restricting yourself to regression to control for covariates can fail if the regression model for outcome, including the treatment effect of interest and the covariates, is incomplete or incorrect. And there's no a priori way to know whether that's the case.

Inverse propensity score weighting provides another way to achieve effective covariate balance between treated and control groups. Cases with a lower probability of getting the treatment get higher weight, providing a more graded balance between treatment groups. That helps to estimate what would have happened had the individuals with the same characteristics been equally represented in control and treatment groups.

You can combine both types of control, via regression and propensity scores, to get what's sometimes called "doubly robust" estimation. If either the regression or the propensity-score model is OK, you can get a reliable measure of treatment effect--provided, as Björn rightly notes in a comment, that there isn't heterogeneity of unobserved covariates affecting outcome between treatment groups.

The issues you raise are much more than a couple of paragraphs can cover. Read the Causal Inference book by Hernán and Robins for a thorough recent treatment.

$\endgroup$
6
  • $\begingroup$ Thank you so much for the explanation and for the link to the book. It is very much appreciated $\endgroup$ Apr 30 at 13:46
  • $\begingroup$ I had "Causality" by Pearl and now I also have the book you suggested. I will start from the one you suggested since it seems newer and also structured $\endgroup$ Apr 30 at 13:56
  • $\begingroup$ +1 You might wish to point out that the Hernán & Robins book has almost a whole chapter devoted to propensity scores (plus other material elsewhere in the book. :) $\endgroup$
    – Alexis
    Apr 30 at 18:29
  • $\begingroup$ "regression [...] can fail if the regression model for outcome [...] is incomplete or incorrect". What do you mean by "incomplete"? If there are confounders we don't observe, propensity score doesn't cure for that, right? Or, if we use propensity score, and match the observations on observed confounders, we hypothesize that the confounders we don't observe do not differ between treatment and control? Is this a valid hypothesis that would make the PS "better" than OLS, which would fail if there is any unobserved confounding no matter what? $\endgroup$ Apr 30 at 22:37
  • 1
    $\begingroup$ I'm accepting this answer because it is comprehensive, well developed, clear, and provide further study materials $\endgroup$ May 1 at 18:04
10
$\begingroup$

Propensity score (PS) analysis has many problems in general, and matching is especially problematic. I prefer covariate adjustment for a spline function of the logit of PS if you need propensity scores, and you must also include pre-specified individual strong covariates to absorb outcome heterogeneity. If the sample size is large in relationship to the number of model parameters, ordinary covariate adjustment without PS works just fine. Problems with PS scores and matching are detailed in links here.

$\endgroup$
8
  • 2
    $\begingroup$ +1 but I think ones needs to clarify how "covariate adjustment without PS" is any different from "regression controlling for the covariates" that the OP suggest. Otherwise we are playing with words. $\endgroup$
    – usεr11852
    Apr 30 at 12:47
  • $\begingroup$ @usεr11852 Yes it would be very useful for me to clarify the difference $\endgroup$ Apr 30 at 13:55
  • $\begingroup$ Yes I meant more standard regression modeling adjusting for covariates including those that would have been in the PS. $\endgroup$ Apr 30 at 17:02
  • 2
    $\begingroup$ Thank you for clarifying. (Sorry for being pushy but I wanted you to say it because people think that "covariance adjustment" is some magical methodology aside their standard regression modelling - which is of course magical in itself.) :) $\endgroup$
    – usεr11852
    Apr 30 at 17:17
  • $\begingroup$ So "ordinary covariate adjustment without PS works just fine" means that my OLS model, with dependent variable = outcome, and independent variable = dummy for treatment + any covariate that (1) co-explains the outcome (2) explains the selection into treatment (is correlated with the treatment dummy), works, also if the distribution of those covariates differ between the treatment group and the control group? $\endgroup$ Apr 30 at 22:28
4
$\begingroup$

Complementary to the the answers from EdM and Frank Harrell (+1 to both).

One might want to consider extensions to using Propensity Scores as the direct probability of treatment group assignment. Usually such work aim to re-weight our sample at hands such that certain features are "balanced". A prime example of that is entropy balancing, (Hainmueller (2012) Entropy Balancing for Causal Effects: A Multivariate Reweighting Method to Produce Balanced Samples in Observational Studies - see the R package ebal). The balancing here refers to using weights such that the moments associated with selected covariates of the two groups of interest are approximately equal (e.g. both groups have similar mean and variance in terms of age and of years of education). There are a few other covariate balancing approach you might want to consider too (e.g. covariate balancing propensity scores (Imai & Ratkovic (2013) Covariate balancing propensity score) or targeted stable balancing weights (Zubizarreta (2015) Stable Weights that Balance Covariates for Estimation With Incomplete Outcome Data) - see the R package CBPS and optweight respectively).

We can use these weights directly or within IPTWS or doubly-robust approach (as EdM suggests). Please note though that no matching method shields us against unmeasured confounding variables.

$\endgroup$
1
  • 1
    $\begingroup$ You can't upweight some observations without downweighting the others, so weighting is statistically inefficient (inflates variance). And the needed procedure to take into account the uncertainty in the weights is complex. $\endgroup$ May 1 at 11:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.