0
$\begingroup$

I'm working on a cohort data, and I'm trying to evaluate the influence from loss to follow-up and missing values. So I performed multiple imputation with chained equation first, then use the imputed datasets to calculated propensity scores (with weightthem function from MatchThem package in r). I followed the tutorial of this package, and evaluated the balance. I got 1.04 in the absolute standardized mean difference, and 0.37 in Kolmogorov-Smirnov statistics, for my propensity score. All the other covariates looks okay. I'm wondering should I proceed with the data to causal effect estimation, or should I do something with this unbalancing?

$\endgroup$

1 Answer 1

0
$\begingroup$

Balance on the propensity score itself is not essential for unbiased estimation of the treatment effect because of the propensity score tautology. The propensity score is a tool used to achieve balance on the covariates, which is what is central. In practice, balance on the covariates usually implies balance on the propensity score, and failure to achieve balance on the propensity score could be a reason for observed imbalance on the covariates. For example, propensity score matching relies on balancing the propensity score extremely closely in order to balance the covariates, but if the covariates are imbalanced, noticing that the propensity score is also imbalanced can indicate that there was a problem in the matching itself (e.g., there aren't enough similar units and a caliper needs to be used or matching must be done with replacement).

Ideally, also, balance on the covariates implies balance on any function of the covariates; the propensity score is a function of the covariates, so some important imbalance on the covariates may remain, even if it is not revealed by the univariate balance statistics.

The solution is to use a better weighting method. It's that simple. The reason WeightIt provides so many weighting methods is to be able to try many and see which yields the best balance. Many do not use a propensity score at all, but are still effective and powerful. You should not be using the default method unless you have a good reason to. Not knowing how to use the others is not a good reason.

$\endgroup$
1
  • $\begingroup$ Thank you Noah, I'm very new to this technique, so I didn't know that different method will greatly change the covariate balance. And this information is very helpful! $\endgroup$
    – YYM17
    Commented Apr 29 at 18:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.