How are propensity scores different from adding covariates in a regression, and when are they preferred to the latter?

Question

I admit I'm relatively new to propensity scores and causal analysis.

One thing that's not obvious to me as a newcomer is how the "balancing" using propensity scores is mathematically different from what happens when we add covariates in a regression? What's different about the operation, and why is it (or is it) better than adding subpopulation covariates in a regression?

I've seen some studies that do an empirical comparison of the methods, but I haven't seen a good discussion relating the mathematical properties of the two methods and why PSM lends itself to causal interpretations while including regression covariates does not. There also seems to be a lot of confusion and controversy in this field, which makes things even more difficult to pick up.

Any thoughts on this or any pointers to good resources/papers to better understand the distinction? (I'm slowly making my way through Judea Pearl's causality book, so no need to point me to that)

Answer 1

One big difference is that regression "controls for" those characteristics in a linear fashion. Matching by propensity scores eliminates the linearity assumption, but, as some observations may not be matched, you may not be able to say anything about certain groups.

For example, if you are studying a worker training program, you may have all the enrollees be men, but the control, non-participant population be composed of men and women. Using regression, you could regress, income, say, on a participation indicator variable and a male indicator. You would use all your data and could estimate the income of a female had she participated in the program.

If you were doing matching, you could only match men to men. As a result, you wouldn't be using any women in your analysis and your results wouldn't pertain to them.

Regression can extrapolate using the linearity assumption, but matching cannot. All the other assumptions are essentially the same between regression and matching. The benefit of matching over regression is that it is non-parametric (except you do have to assume that you have the right propensity score, if that is how you are doing your matching).

For more discussion, see my page here for a course that was heavily focused on matching methods. See especially Causal Effects Estimation Strategy Assumptions.

Also, be sure to check out the Rosenbaum and Rubin (1983) article that outlines propensity score matching.

Lastly, matching has come a long way since 1983. Check out Jas Sekhon's webpage to learn about his genetic matching algorithm.

Answer 2

A likely obtuse reference, but if you by chance have access to it I would recommend reading this book chapter (Apel and Sweeten, 2010). It is aimed at social scientists and so perhaps not as mathematically rigorous as you seem to want, but it should go into enough depth to be more than a satisfactory answer to your question.

There are a few different ways people treat propensity scores that can result in different conclusions from simply including covariates in a regression model. When one matches scores one does not necessarily have common support for all observations (i.e. one has some observations that appear to never have the chance to be in the treatment group, and some that are always in the treatment group). Also one can weight observations in various ways that can result in different conclusions.

In addition to the answers here I would also suggest you check out the answers to the question chl cited. There is more substance behind propensity scores than simply a statistical trick to achieve covariate balance. It you read and understand the highly cited articles by Rosenbaum and Rubin it will be more clear why the approach is different than simply adding in covariates in a regression model. I think a more satisfactory answer to your question is not necessarily in the math behind propensity scores but in their logic.

Answer 3

The short answer is that propensity scores are not any better than the equivalent ANCOVA model, particularly with regard to causal interpretation.

Propensity scores are best understood as a data reduction method. They are an effective means to reduce many covariates into a single score that can be used to adjust an effect of interest for a set of variables. In doing so, you save degrees of freedom by adjusting for a single propensity score rather than multiple covariates. This presents a statistical advantage, certainly, but nothing more.

One question which may arise when using regression adjustment with propensity scores is whether there is any gain in using the propensity score rather than performing a regression adjustment with all of the covariates used to estimate the propensity score included in the model. Rosenbaum and Rubin showed that the "point estimate of the treatment effect from an analysis of covariance adjustment for multivariate X is equal to the estimate obtained from a univariate covariance adjustment for the sample linear discriminant based on X, whenever the same sample covariance matrix is used for both the covariance adjustment and the discriminant analysis". Thus, the results from both methods should lead to the same conclusions. However, one advantage to performing the two-step procedure is that one can fit a very complicated propensity score model with interactions and higher order terms first. Since the goal of this propensity score model is to obtain the best estimated probability of treatment assignment, one is not concerned with over-parameterizing this model.

From:

PROPENSITY SCORE METHODS FOR BIAS REDUCTION IN THE COMPARISON OF A TREATMENT TO A NON-RANDOMIZED CONTROL GROUP

D'Agostino (quoting Rosenbaum and Rubin)

http://web.pdx.edu/~nwallace/EPA/Dagostino1998.pdf

Answer 4

I like to think of PS as a design portion of the study which completely separated from the analysis. That is, you might want to think in terms of design (PS) and analysis (regression etc...). Also, PS porvides a mean of supporting exchangeability for binary treatment; maybe others can comment on whether including the covariates in the outcome model can actaully support exchangebility, or whether one assume exchangeability prior to including the covariates in the outcome model.