# How are propensity scores different from adding covariates in a regression, and when are they preferred to the latter?

I admit I'm relatively new to propensity scores and causal analysis.

One thing that's not obvious to me as a newcomer is how the "balancing" using propensity scores is mathematically different from what happens when we add covariates in a regression? What's different about the operation, and why is it (or is it) better than adding subpopulation covariates in a regression?

I've seen some studies that do an empirical comparison of the methods, but I haven't seen a good discussion relating the mathematical properties of the two methods and why PSM lends itself to causal interpretations while including regression covariates does not. There also seems to be a lot of confusion and controversy in this field, which makes things even more difficult to pick up.

Any thoughts on this or any pointers to good resources/papers to better understand the distinction? (I'm slowly making my way through Judea Pearl's causality book, so no need to point me to that)

-
Recommend you read Morgan and Winship, 2007. Chapters 4 and 5 do an explicit compare and contrast of regression and matching for causal effect identification. – conjugateprior Apr 9 '12 at 19:36
+1 on Morgan and Winship. – Dimitriy V. Masterov Apr 9 '12 at 21:41

One big difference is that regression "controls for" those characteristics in a linear fashion. Matching by propensity scores eliminates the linearity assumption, but, as some observations may not be matched, you may not be able to say anything about certain groups.

For example, if you are studying a worker training program, you may have all the enrollees be men, but the control, non-participant population be composed of men and women. Using regression, you could regress, income, say, on a participation indicator variable and a male indicator. You would use all your data and could estimate the income of a female had she participated in the program.

If you were doing matching, you could only match men to men. As a result, you wouldn't be using any women in your analysis and your results wouldn't pertain to them.

Regression can extrapolate using the linearity assumption, but matching cannot. All the other assumptions are essentially the same between regression and matching. The benefit of matching over regression is that it is non-parametric (except you do have to assume that you have the right propensity score, if that is how you are doing your matching).

For more discussion, see my page here for a course that was heavily focused on matching methods. See especially Causal Effects Estimation Strategy Assumptions.

Also, be sure to check out the Rosenbaum and Rubin (1983) article that outlines propensity score matching.

Lastly, matching has come a long way since 1983. Check out Jas Sekhon's webpage to learn about his genetic matching algorithm.

-
 Maybe this is because I am not a statistician, but when it seems you assumed linear regression when the OP asked about regression in general. But I guess the gist is that adding covariates to any kind of regressor makes some assumptions about the input space so it can extrapolate to new examples, and matching is more cautious about what kind of things can be extrapolated. – rrenaud Apr 9 '12 at 19:37

A likely obtuse reference, but if you by chance have access to it I would recommend reading this book chapter (Apel and Sweeten, 2010). It is aimed at social scientists and so perhaps not as mathematically rigorous as you seem to want, but it should go into enough depth to be more than a satisfactory answer to your question.

There are a few different ways people treat propensity scores that can result in different conclusions from simply including covariates in a regression model. When one matches scores one does not necessarily have common support for all observations (i.e. one has some observations that appear to never have the chance to be in the treatment group, and some that are always in the treatment group). Also one can weight observations in various ways that can result in different conclusions.

In addition to the answers here I would also suggest you check out the answers to the question chl cited. There is more substance behind propensity scores than simply a statistical trick to achieve covariate balance. It you read and understand the highly cited articles by Rosenbaum and Rubin it will be more clear why the approach is different than simply adding in covariates in a regression model. I think a more satisfactory answer to your question is not necessarily in the math behind propensity scores but in their logic.

-
 +1 for this well-balanced answer. – chl♦ Mar 22 '11 at 13:18 @Andy W See the quote from Rosenbaum and Rubin on the equivalence of regression with covariates and propensity score adjustment in my updated post. – Brett Magill Apr 9 '12 at 21:39

The short answer is that propensity scores are not any better than the equivalent ANCOVA model, particularly with regard to causal interpretation.

Propensity scores are best understood as a data reduction method. They are an effective means to reduce many covariates into a single score that can be used to adjust an effect of interest for a set of variables. In doing so, you save degrees of freedom by adjusting for a single propensity score rather than multiple covariates. This presents a statistical advantage, certainly, but nothing more.

One question which may arise when using regression adjustment with propensity scores is whether there is any gain in using the propensity score rather than performing a regression adjustment with all of the covariates used to estimate the propensity score included in the model. Rosenbaum and Rubin showed that the "point estimate of the treatment effect from an analysis of covariance adjustment for multivariate X is equal to the estimate obtained from a univariate covariance adjustment for the sample linear discriminant based on X, whenever the same sample covariance matrix is used for both the covariance adjustment and the discriminant analysis". Thus, the results from both methods should lead to the same conclusions. However, one advantage to performing the two-step procedure is that one can fit a very complicated propensity score model with interactions and higher order terms first. Since the goal of this propensity score model is to obtain the best estimated probability of treatment assignment, one is not concerned with over-parameterizing this model.

From:

PROPENSITY SCORE METHODS FOR BIAS REDUCTION IN THE COMPARISON OF A TREATMENT TO A NON-RANDOMIZED CONTROL GROUP

D'Agostino (quoting Rosenbaum and Rubin)

http://web.pdx.edu/~nwallace/EPA/Dagostino1998.pdf

-
(+1) There was also an interesting thread about the causality issue in this related question, From a statistical perspective, can one infer causality using propensity scores with an observational study?. – chl Mar 22 '11 at 7:36
I agree with the general premise of this answer, but when one matches based on the propensity scores it is not the same as plopping all the covariates into the model (and hence is not just a dimension reduction technique). It is not the same if one weights by propensity scores either. – Andy W Mar 22 '11 at 12:46
@ Andy. Indeed. That is an important point and my answer referred to propensity score adjustment. Stratifying and matching will produce different results that the ANCOVA model and can have some undesirable properties (which you hint at in your reply below). – Brett Magill Mar 22 '11 at 14:02
@Andy. I just noticed that last sentence. Adjusting for a propensity score and ANCOVA models produce equivalent estimates of effects. I've read results of simulations that show this clearly. I agree, matching and stratifying are different animals, but propensity score adjustment and the equivalent ANCOVA will get you the same effect estimate. You get more power via the reduction of covariates, but that's it. – Brett Magill Mar 22 '11 at 14:05
I don't know the correct lingo, but if when you say adjustment you mean including the propensity score on the RHS of a regression equation then yes I agree that it is equivalent to the ANCOVA solution. I'm not quite sure what undesirable properties I hinted at, although they definitely exist (that may be getting off topic though for this question). – Andy W Mar 22 '11 at 14:31