1
$\begingroup$

This query is cross posted from the Statalist forum where it did not prompt any responses.

I am trying to understand whether an intervention ("tha") is associated with increased 12-month mortality ("rip") using StataSE 13.0. My first step was to match the groups across a number of variables ("age", "sex", "preopasa", "premob", and "origin") using the "cem" package for coarsened exact matching. My understanding from the literature around CEM is that researchers should then continue with their analyses (e.g. multivariable regression) as normal but using the matched groups.

When I run the matching code:

ssc install cem
cem age sex preopasa premob origin, treatment(tha) autocuts(fd)

It appears to work and almost all patients (29,181/29,267) are allocated to 56 matched strata. Stata creates a number of new variables: cem_strata, cem_matched, and cem_weights, which seems to be what is expected.

To my non-statistician mind, I imagined that that those co-variables would then be become less significant in any subsequent regression models. However, this doesn’t appear to be the case.

When I run code that I believe should run a logistic regression model using the matched weights:

logistic rip age sex preopasa premob origin tha [iweight=cem_weights]

Logistic regression using matched cohort

I get an output that is barely any different – and in some cases shows bigger odds ratios / wider confidence intervals – than when I run code without the CEM weights:

logistic rip age sex preopasa premob origin tha

Logistic regression without including CEM weights

I think that I have used the cem package correctly (at least following the package instructions) but wonder whether I am right to be concerned that there is still so much residual confounding in the (apparently) matched dataset.

Can anyone spot what might have gone wrong and/or suggest a way of formally examining whether or not the match process worked?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

Here is the key intuition: balancing your variables does not affect their relationship with the outcome. If the coefficient of X on Y is 3 in the population, it will be 3 in the matched group as well. The point of using matching is that if there is actually a curvilinear relationship in the population, a linear approximation will perform better in the matched sample than in the whole sample, which means you don't need to get the functional form of the relationships between your covariates and the outcome correct to isolate your treatment effect; ideally, you won't even need to include the covariates in your outcome model to get an unbiased estimate of the treatment effect.

After matching, what you should see is that the relationships between the covariates and the treatment disappears. If your covariates are still related to the treatment after matching, your matching has failed, and you should try another method.

So what you should check is whether the relationship between your covariates changes between the matched sample and the full sample. If the relationship weakens, then great; you have reduced confounding and basically any outcome model will give you an unbiased estimate of the treatment effect. You can also verify this yourself: does removing a covariate from the outcome model change the treatment effect in the matched sample? (It shouldn't.) What about in the unmatched sample? (It should if that variable is highly related to both treatment and outcome).

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ Did the matching result in discarding any observations? If so this is not good statistical practice and is inefficient. And in contexts such as the current one, matching is unlikely to compete with covariable adjustment. Matching hides interactions with treatment and results in residual heterogeneity when matching on a continuous variable with a non-fine caliper. I'm not getting the original motivation for matching here. $\endgroup$
    – Frank Harrell
    Mar 17, 2018 at 12:08
  • $\begingroup$ OP mentioned that all but ~100 observations were retained. As Gay King has written extensively about (e.g., Ho, Imai, King & Stuart, 2007; Nielson & King, 2016), the point of matching, even if you discard observations, is to reduce the dependency on your getting your parametric model correct to isolate the treatment effect. It's not poor statistical practice to discard observations that are not relevant to identifying the causal effect of interest. $\endgroup$
    – Noah
    Mar 18, 2018 at 2:55
  • $\begingroup$ Also, the loss of efficiency isn't guaranteed; if those observations you discard have outcomes that are very far from the outcomes of those you retain, which will be the case if you discard observations that have distant covariate values, which you do when matching in this way, you can improve efficiency by reducing your MSE. $\endgroup$
    – Noah
    Mar 18, 2018 at 2:57
  • 1
    $\begingroup$ The observations you are deleted are relevant to the causal effect of interest. But you are right to bring in the assumption about adequate model fit. That's why I make heavy use of flexible additive models using splines for continuous variables. And the loss of efficiency you mention from covariate adjustment only happens if there are omitted interaction terms in the model. In addition everyone is avoiding precisely how to do a proper matched analysis, which is far from clear. $\endgroup$
    – Frank Harrell
    Mar 18, 2018 at 13:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.