Regression after matching

Question

I performed 1:1 nearest neighbor matching on 16 covariates using matchit package on R. Covariate balance looks good for most covariates, but there are some that looks less than ideal. I then ran regression on matched dataset to estimate the impact by "hopefully" adjusting for remaining differences in covariates.

My question is - how do I know regression has sufficiently controlled for remaining differences in covariate values after matching?

Thanks in advance for any pointers!

Answer 1

In the same way we don't know the form of the outcome model (which is why we use propensity score matching in the first place), we don't know whether regression completely removes all confounding in a matched sample. Matching makes it more plausible for confounding to be removed by regression; this is the main thesis of Ho, Imai, King, and Stuart (2007), the paper that motivated the development of MatchIt.

You should do as much work as you can in the matching phase to ensure bias is eliminated by assessing balance on the covariates not just on their means, but on their entire distributions and their interactions. The cobalt package provides many diagnostics for doing so and works with MatchIt objects.

You should also do as much work as you can to flexibly model the outcome; although main effects regression of the outcome of the covariates and treatment may be sufficient, you can be more certain of the elimination of bias by using a flexible model, such as a generalized additive model, regression model with splines, or machine learning method like Bayesian additive regression trees (BART) or generalized boosted modeling (GBM; not the twang implementation). If either the matching is sufficient to eliminate imbalance or the outcome model is correct, the effect estimate will have low bias; this property is known as "double-robustness".