6
$\begingroup$

I had a question about matching. I understand the benefits of matching prior to conducting a study due to potential increases in statistical efficiency/ adjustment for confounders. Let's say you're conducting a retrospective study where you have a few people exposed to some chemical, perhaps 5 or so, and you already have data on 100,000+ controls, perhaps through some hospital EHR system. Is there a reason to match exposed to unexposed 1:1 or even 1:10 vs. just using all 100,000+ controls that you have?

Specifically, I've seen people do something similar using propensity score matching to address confounders, but I'm not understanding why people throw away control data rather than just including the confounders as some covariates in a model (and account for nonlinearities in other ways).

$\endgroup$

1 Answer 1

5
$\begingroup$

This is an ongoing debate with no universal solution. Many advocate that you should just do as you suggested and simply regress the outcome on the covariates and treatment. If the regression model is correct, confounding is eliminated and no data has to be thrown out. The benefits of matching are that, in some cases, you can control for confounding in a more robust way. Exact, near exact, or coarsened exact matching allow you to control not just for the linear terms of the covariates, but for any arbitrary combination of them regardless of the form of the outcome model. With massive datasets like the one you mention, matching in this way can be very effective at eliminating bias without losing much precision, even if many units are thrown away.

There are points at which having more control units does not improve the precision of the estimates. For example, 10:1 matching is not much better than 5:1 matching (Rosenbaum, 2020). Because of this, discarding control units is not problematic when the treatment group is large and one is able to find good matches for each treated unit. In the matched sample, a regression of the outcome on the covariates and treatment (i.e., combining matching and regression) can yield improved precision and bias reduction than either alone.

The state of matching in the contemporary applied literature is grim. Propensity score matching still dominates despite many other superior alternatives. It is often seen as a default even though in many cases regression, if done with care, could yield far more precise and equally unbiased estimates. Other alternatives, like BART and TMLE (which I briefly explain here), have been demonstrated to be superior in many cases, and because of their robustness, they should be viewed with more trust than propensity score matching in my opinion. Frank Harrell has done quite a bit of research demonstrating the effectiveness of regression and methods to use it optimally.

There are some times when matching can generally be preferred to regression. Some studies have found that with few events, logistic regression in a matched sample performs better than logistic regression including the covariates in the model because of the events-per-variable problem. When there are extreme nonlinearities in the effect modification by some covariates, matching on them can make it easier to estimate a simpler, well-fitting regression to the subset of the data where the causal contrast is most relevant (see Ho et al., 2007, for an example).

I suppose the short answer is that matching, if done right, can yield highly robust and unbiased treatment effect estimates, which is what inspired its early use and continued development, but propensity score matching as it is often done today is suboptimal compared to more sophisticated methods that exist and perform better more generally. Although regression uses all the available units, there is a point at which having more control units doesn't improve the precision of the estimate, in which case, if removing some units decreases the bias and improves the robustness of an estimate, performing regression in the matched sample may be just as, if not more, effective than simply performing regression in the full sample.

An investigation into the comparative performance of matching and regression, both done as optimally as possible, needs to be done, in my opinion, as your question is a valid one and commonly asked. I think causal inference researchers (myself included) need to do a better job of clearly articulating when matching should be preferred to regression and how to best perform matching using the statistical advances that have occurred in the last ten years.


Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference. Political Analysis, 15(3), 199–236. https://doi.org/10.1093/pan/mpl013

Rosenbaum, P. R. (2020). Modern Algorithms for Matching in Observational Studies. Annual Review of Statistics and Its Application, 7(1), 143–176. https://doi.org/10.1146/annurev-statistics-031219-041058

$\endgroup$
4
  • $\begingroup$ Amazing answer! Thank you. $\endgroup$ Mar 24, 2020 at 20:58
  • $\begingroup$ Followup question: Can a matching method (such as coarsened exact matching) which, after matching on $k$ variables, yields treatment and control groups that are balanced on the remaining $p-k$ variables, still produce biased ATE estimates (which a different method like TMLE would correct for)? Or is it not possible to beat CEM in this scenario, even if it does take some time(!) to search for the right combination of matching variables to achieve balanced trmt and ctrl groups? $\endgroup$
    – RobertF
    Apr 24, 2020 at 16:46
  • $\begingroup$ That's a good question, but it deserves its own post. $\endgroup$
    – Noah
    Apr 24, 2020 at 18:25
  • $\begingroup$ Thanks - I just posted the question here on Cross Validated. $\endgroup$
    – RobertF
    Apr 24, 2020 at 20:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.