19
$\begingroup$

Propensity score matching is used for make causal inferences in observational studies (see the Rosenbaum / Rubin paper). What's the simple intuition behind why it works?

In other words, why if we make sure the probability of participating in the treatment is equal for the two groups, the confounding effects disappear, and we can use the result to make causal conclusions about the treatment?

$\endgroup$
4
  • 8
    $\begingroup$ Mine is probably an unpopular opinion, but I think saying that PSM allows you to make causal inferences is stretching it a bit. Yes, it is a finer matching method than just "crude" matching on a handful of variables, but at the end of the day, you're still only matching on only those variables that are observable to you. You might still have endogenous treatment, but you make the assumption that after matching, treatment is exogenous. But then again, I'm a nobody compared to the likes of Rubin :) $\endgroup$ Apr 12, 2016 at 0:29
  • 5
    $\begingroup$ There are those who disagree with Rosenbaum and Rubin. Gary King has argued pretty effectively that the results from matching using PSM are inferior to the results obtained from matching based on Mahalanobis distance. See his paper here ... gking.harvard.edu/files/gking/files/psnot.pdf?m=1456683191 In addition, this webinar provides even more compelling evidence ... methods-colloquium.com/… $\endgroup$ Apr 12, 2016 at 1:10
  • 4
    $\begingroup$ In theory, you CAN make causal inference applying Rubin's causal model if all assumptions are met. But of course the devil is always in the details of the assumption, and for me, the assumption of unobserved confounders is often a tough one. In fact, it's impossible to prove. But you can at least perform sensitivity analyses to determine what it would take for your conclusions to change if you had unobserved confounders. $\endgroup$ Apr 12, 2016 at 1:10
  • 5
    $\begingroup$ Good point @DJohnson. There are also the likes of Judea Pearl, Ian Shrier, and Arvid Sjolander that are are critical of the Rubin's Potential Outcomes Framework for causal inference due to the possibility of introducing "M-bias" through what they call a "collider." They are interesting reads and should supplement any readings from Rubin. $\endgroup$ Apr 12, 2016 at 1:13

4 Answers 4

17
$\begingroup$

I'll try to give you an intuitive understanding with minimal emphasis on the mathematics.

The main problem with observational data and analyses that stem from it is confounding. Confounding occurs when a variable affects not only the treatment assigned but also the outcomes. When a randomized experiment is performed, subjects are randomized to treatments so that, on average, the subjects assigned to each treatment should be similar with respect to the covariates (age, race, gender, etc.). As a result of this randomization, it's unlikely (especially in large samples) that differences in the outcome are due to any covariates, but due to the treatment applied, since, on average, the covariates in the treatment groups are similar.

On the other hand, with observational data there is no random mechanism that assigns subjects to treatments. Take for example a study to examine the survival rates of patients following a new heart surgery compared to a standard surgical procedure. Typically one cannot randomize patients to each procedure for ethical reasons. As a result patients and doctors self-select into one of the treatments, often due to a number of reasons related to their covariates. For example the new procedure might be somewhat riskier if you are older, and as a result doctors might recommend the new treatment more often to younger patients. If this happens and you look at survival rates, the new treatment might appear to be more effective, but this would be misleading since younger patients were assigned to this treatment and younger patients tend to live longer, all else being equal. This is where propensity scores come in handy.

Propensity scores helps with the fundamental problem of causal inference -- that you may have confounding due to the non-randomization of subjects to treatments and this may be the cause of the "effects" you are seeing rather than the intervention or treatment alone. If you were able to somehow modify your analysis so that the covariates (say age, sex, gender, health status) were “balanced” between the treatment groups, you would have strong evidence that the difference in outcomes is due to the intervention/treatment rather than these covariates. Propensity scores, determine each subject’s probability of being assigned to the treatment that they received given the set of observed covarites. If you then match on these probabilities (propensity scores), then what you have done is taken subjects who were equally likely to be assigned to each treatment and compared them with one another, effectively comparing apples to apples.

You may ask why not exactly match on the covariates (e.g. make sure you match 40 year old men in good health in treatment 1 with 40 year old men in good health in treatment 2)? This works fine for large samples and a few covariates, but it becomes nearly impossible to do when the sample size is small and the number of covariates is even moderately sized (see the curse of dimensionality on Cross-Validated for why this is the case).

Now, all this being said, the Achilles heel of propensity score is the assumption of no unobserved confounders. This assumption states that you have not failed to include any covariates in your adjustment that are potential confounders. Intuitively, the reason behind this is that if you haven’t included a confounder when creating your propensity score, how can you adjust for it? There are also additional assumptions such as the stable unit treatment value assumption, which states that the treatment assigned to one subject does not affect the potential outcome of the other subjects.

$\endgroup$
9
$\begingroup$

In a strict sense, propensity score adjustment has no more to do with causal inference than regression modeling does. The only real difference with propensity scores is that they make it easier to adjust for more observed potential confounders than that sample size may allow regression models to incorporate. Propensity score adjustment (best done through covariate adjustment in the majority of cases, using a spline in the logit PS) can be thought of as a data reduction technique where the reduction is along an important axis - confounding. It does not however handle outcome heterogeneity (susceptibility bias) so you also have to adjust for key important covariates even when using propensities (see also issues related to non-collapsibility of odds and hazard ratios).

Propensity score matching can exclude many observations and thus be terribly inefficient. I view any method that excludes relevant observations as problematic. The real problem with matching is that it excludes easily matched observations due to some perceived need for having 1:1 matching, and most matching algorithms are observation order-dependent.

Note that it is very easy when doing standard regression adjustment for confounding to check for and exclude non-overlap regions. Propensity score users are taught to do this and the only reason regression modelers don't is that they are not taught to.

Propensity score analysis hides any interactions with exposure, and propensity score matching hides in addition a possible relationship between PS and treatment effect.

Sensitivity (to unmeasured confounders) analysis has been worked out for PS but is even easier to do with standard regression modeling.

If you use flexible regression methods to estimate the PS (e.g., don't assume any continuous variables act linearly) you don't even need to check for balance - there must be balance or the PS regression model was not correctly specified in the beginning. You only need to check for non-overlap. This assumes there are no important interactions that were omitted from the propensity model. Matching makes the same assumption.

$\endgroup$
4
$\begingroup$

I recommend checking out Mostly Harmless Econometrics - they have a good explanation of this at an intuitive level.

The problem you're trying to solve is selection bias. If a variable $x_i$ is correlated with the potential outcomes $y_{0i},y_{1i}$ and with the likelihood of receiving treatment, then if you find that the expected outcome of the treated is better than the expected outcome of the untreated, this may be a spurious finding since the treated tend to have higher $x$ and therefore have higher $y_{0i},y_{1i}$. The problem arises because $x$ makes $y_{0i},y_{1i}$ correlated with the treatment.

This problem can be solved by controlling for $x$. If we think that the relationship between the potential outcomes and the variables $x$ is linear, we just do this by including $x$ in a regression with a dummy variable for treatment, and the dummy variable interacted with $x$. Of course, linear regression is flexible since we can include functions of $x$ as well. But what if we do not want to impose a functional form? Then we need to use a non-parametric approach: matching.

With matching, we compare treated and untreated observations with similar $x$. We come away from this with an estimate of the effect of treatment for all $x$ values (or small ranges of values or "buckets") for which we have both treated and untreated observations. If we do not have many such $x$ values or buckets, in particular if $x$ is a high-dimensional vector so it is difficult to find observations close to one other, then it is helpful to project this space onto one dimension.

This is what propensity score matching does. If $y_{0i},y_{1i}$ are uncorrelated with treatment given $x_i$, then it turns out that they are also uncorrelated with treatment given $p(x_i)$ where $p(x)$ is the probability of treatment given $x$, i.e. the propensity score of $x$.

Here's your intuition: if we find a sub sample of observations with a very similar propensity score $p(x)$, then for that sub-sample, the treated and untreated groups are uncorrelated with $x$. Each observation is equally likely to be treated or untreated; this implies that any treated observation is equally likely to come from any of the $x$ values in the sub-sample. Since $x$ is what determines the potential outcomes in our model, this implies that, for that sub-sample, the potential outcomes $y_{0i},y_{1i}$ are uncorrelated with the treatment. This condition ensures that the sub-sample average difference of outcome between treated and untreated is a consistent estimate of the average treatment effect on this sub-sample, i.e.

$$ E[y_i|\text{Treated},p(x)] - E[y_i|\text{Untreated},p(x)] $$

is a consistent estimate of the local average treatment effect.

Further reading:

Should we really use propensity score matching in practice?

Related question comparing matching and regression

$\endgroup$
1
$\begingroup$

It "works" for the same reason that regression "works" - you're controlling for all confounding factors.

You can accomplish such analytical control by a fully specified regression model with perhaps many confounding variables, or a regression model with only one variable - the propensity score (that may or may not be an equally complicated model consisting of those same confounders). You could stick with this regression vs the propensity score, or you could compare the response within similar groups, where similarity is defined by the propensity score. In spirit you're doing the same thing, but some people feel that the latter method better highlights the causal task at hand.

Update following feedback

My thought for explaining the intuition behind why propensity score matching works was to explain the Propensity Score Theorem, i.e., $$Y(0), Y(1) \perp T \, | \, X \Rightarrow Y(0), Y(1) \perp T \, | \, p(X),$$ something I thought I could do using regression. But as @StatsStudent argues, regression makes it easy to extrapolate comparisons between treatment and control that never occur in the data. If this is part of why propensity score matching "works," then my answer was incomplete. I consulted Counterfactuals and Causal Inference and read about one version of nearest-neighbor matching, called "caliper matching" (p. 108) where propensity scores of treatment and nearest control case must be within some maximum distance, resulting in some treatment cases without matches. In this case, the method would still work by adjusting for the propensity score using a nonparametric analogue to regression, but it also makes clear what can't be known from the data alone (without a model to extrapolate from) and allowing a redefinition of the causal quantity given the available data.

$\endgroup$
2
  • 1
    $\begingroup$ This answer is destined to be disliked because it cheapens propensity score methods by comparing them to regression, a dirty word in causal circles. But the leading answer admits that "the Achilles heel of propensity score is the assumption of no unobserved confounders." That's a rather big Achilles heel. Say the assumption is met and I've got a data set with all the confounders in the universe. Help me understand why regression won't work to uncover the causal truth but propensity score matching will. $\endgroup$
    – Ben Ogorek
    Apr 16, 2016 at 0:33
  • 1
    $\begingroup$ The reason why regression is often problematic is because results from regression are extrapolated. With propensity scores, one ensures, during covariate balance checks that there is sufficient confounder overlap between the treatment groups. This is not necessarily the case in regression & can lead to inaccurate estimates as there are no standard diagnostic checks that will tell you your regression model is extrapolating. That's why I typically run propensity score balancing checks even if I'm just running a regression model and don't plan to use the propensity scores themselves. $\endgroup$ Apr 16, 2016 at 2:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.