3
$\begingroup$

I am new to the fascinating world of matching and propensity scoring. It is highly likely that I will be using some (or more) matching method(s) for my forthcoming project, probably with the R package MatchIt.

I am particularly interested in finding out whether there is a method that makes it possible to interpret the regression coefficients of covariates following matching. In math terms, say you have an outcome variable Y, a treatment variable A and a couple of covariates X1 and X2. So you first match with a model that looks something like this: A ~ X1 + X2 (I'm using the R language notation here) and then use the matched dataset to run a regression like Y ~ A + X1 + X2. I would like to be able to interpret the coefficients of this model for X1 and X2.

The relevant peer-reviewed work (Ho et al., 2007, 2011) and vignettes (Greifer, 2022) have confused me a bit. It looks like the school of thought represented by Ho et al. is that matching can be used for preprocessing data before applying some model with Y as the outcome and A, X1 and X2 as the independent variables (like the model above). To me, this is like saying that, after matching the data, I can use whatever model, e.g. linear regression, GLM, GAM, Random Forest etc. to interpret the relationship of Y with X1 and X2 through e.g. regression coefficients, partial plots etc. (See also How exactly to evaluate Treatment effect after Matching? and Regression after matching .)

On the other hand, Greifer (2022) states, in many places, that one should not interpret the covariate coefficients:

It is important not to interpret the coefficients and tests of the other covariates in the outcome model. These are not causal effects and their estimates may be severely confounded. Only the treatment effect estimate can be interpreted as causal assuming the relevant assumptions about unconfoundedness are met. Inappropriately interpreting the coefficients of covariates in the outcome model is known as the Table 2 fallacy [...]

So it is not clear to me if I can or cannot interpret the coefficients of the covariates following matching. I am probably missing something, so some clarity would be highly appreciated!

References Greifer N (2022). Estimating Effects After Matching. https://kosukeimai.github.io/MatchIt/articles/estimating-effects.html#after-pair-matching-without-replacement [accessed 11 Jan 2022]

Ho DE, Imai K, King G & Stuart EA (2007). Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric Causal Inference. Political Analysis 15:199–236. doi:10.1093/pan/mpl013

Ho DE, Imai K, King G & Stuart EA (2011). MatchIt: Nonparametric Preprocessing for Parametric Causal Inference. Journal of Statistical Software 42(8).

$\endgroup$

1 Answer 1

1
$\begingroup$

Because matching concerns a focal treatment variable, the nonparametric preprocessing only reduces the model dependence of the effect of that variable on the outcome, but not of other variables on the outcome. The (intended) result of matching is to eliminate the association between the treatment and the covariates, but the relationships among the covariates remain. So, not all predictors are treated equally in the nonparametric preprocessing; the treatment is given a special status that allows its coefficient not to depend on how the outcome model is specified. I discuss this a bit in this answer.

That is separate from the issue of whether you should interpret the coefficients of covariates at all. The answer to this is "no", and has nothing to do with matching. This is covered extensively in Westreich and Greenland (2013) on the "table 2" fallacy. The reason interpreting the non-focal coefficients causal is so problematic is that the relationship between the covariates and the outcome may be totally confounded or suffer from collider bias. We absolutely know that, at best, they can only be interpreted as conditional relationships (i.e., conditional on the treatment, which mediates the relationship between the covariates and the outcome), but their effects may be confounded, blocked, or otherwise biased beyond interpretation.

So, I will reiterate with certainty that you should not interpret the coefficients on covariates in a regression model fit after matching; only the treatment variable is interpretable as a total, covariate-adjusted effect on the outcome. The literature on nonparametric preprocessing is written with this context already in mind (i.e., with the focus only on a single focal causal variable), which is why it may be under-addressed there.

$\endgroup$
5
  • $\begingroup$ Thanks! Having read W&G I wonder if the best approach to the issue of interpreting covariate coefficients (before/without matching) is to design a modelling process that follows a few necessary protocols: 1. Make causal diagram; 2. Remove colliders; 3. Fit model. 4. Make clear the distinction between direct and other effects; 5. Do sensitivity analysis for any unmeasured covariates (there must be an R package for this!). I find it quite attractive that with a covariate adjusted model one could potentially study the effect of all covariates- it could bring so much insight. Any thoughts? $\endgroup$
    – andmoj
    Commented Jan 13, 2022 at 9:13
  • $\begingroup$ All the work you do to de-bias the treatment effect estimate using matching you would have to do for each covariate in the model. In no way can you just read off the coefficients from a single regression and interpret them causally. A problem is that the treatment mediates the covariate-outcome relationship, but different treatments would mediate those relationships differently, so from a model that evaluates a single treatment you cannot learn anything about the general covariate effects. You can always just fit several other models but at that point you're doubling the work. $\endgroup$
    – Noah
    Commented Jan 13, 2022 at 15:19
  • $\begingroup$ I'm convinced! A huge thank you for your time! $\endgroup$
    – andmoj
    Commented Jan 14, 2022 at 9:29
  • $\begingroup$ A useful resource: dagitty.net/learn/graphs/table2-fallacy.html $\endgroup$
    – andmoj
    Commented Jan 14, 2022 at 9:29
  • $\begingroup$ @andmoj that is a great resource! $\endgroup$
    – Noah
    Commented Jan 20, 2022 at 19:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.