1
$\begingroup$

I have a dataset of patients with a grouping variable (groups A (control) and group B (treatment)). The two groups have sample sizes of 170 vs. 30. I would like to compare outcomes between the two treatment groups but they differ in baseline covariates. I tried propensity score matching and inverse probability treatment weights (IPTW) but both don't seem to achieve good covariate balance between the groups (as of SMD <0.1).

My guess is the sample size of the treatment group is too small. Does anybody have a recommendation how I can adjust for baseline covariates in such a small sample size?

$\endgroup$
3
  • $\begingroup$ Can you please expain (as an edit) the abbrevs IPTW, PS, etc? $\endgroup$ Dec 13, 2021 at 23:51
  • $\begingroup$ Did you do the most basic versions of propensity score matching and IPTW, or did you do advanced versions that target balance like overlap weighting, cardinality matching, genetic matching, and entropy balancing? Is there a reason you don't want to adjust for covariates using regression? $\endgroup$
    – Noah
    Dec 13, 2021 at 23:59
  • $\begingroup$ Yes, I did the most basic versions of PS match and IPTW. And there is no big reason why not to do regression. $\endgroup$
    – Takanashi
    Dec 14, 2021 at 7:46

2 Answers 2

2
$\begingroup$

Differences in baseline covariates are not intrinsically a bad thing. Given the small sample size, and the possible number of "covariates" compared, it's almost a certainty one or more will show "imbalance" - but that's just an artifact of an uncontrolled familywise error rate - and hey, what's the primary endpoint anyway?

It's most useful to think of the randomization assumption not as balancing a possibly infinite list of covariates, but balancing their "average effect" on each group; exactly the rationale for propensity matching. Stephen Senn has written extensively on this issue as one of the major "myths" of clinical trials - and we must infer by the convention of "treated" and "control" that this is a randomized study. https://onlinelibrary.wiley.com/doi/abs/10.1002/sim.5713

Dr. Senn goes on the explain that imbalance is much different from confounding, since even if the variable is associated with the outcome, it is theoretically unrelated to randomization, unless there is some kind of conditional analysis such as in the "per protocol" population. In spite of all this, adjustment prevails as the superior way to handle imbalance because, given the variable conditioned upon is prognostic - i.e. it actually predicts the outcome - the tendency is to increase power. Variables that are not strongly predictive of the response tend to attenuate the power of the test when adjusted willy nilly.

But all of this goes in the bin if it's not actually a prespecified analysis. This is because you can't choose the analysis that you finally do based upon what you see in the data. Do the analysis you planned, it's probably the right one.

$\endgroup$
1
$\begingroup$

Look to see if any of these imbalanced covariates is associated with the outcome variable through regression on the available data or via a literature review. If there is little evidence to suggest they are associated, then you would feel that much more confident there is no harm in having the imbalance. You may end up choosing to balance only one or two of the covariates that indeed show evidence of an association with the endpoint. Ultimately you sill still need the "no unmeasured confounders" preface if you are going to make causal inference. See this thread.

You could perform several sensitivity analyses to see the effect of balancing each covariate separately and in combination, sample size permitting.

Also look at this thread regarding imbalance in patient characteristics due to post-baseline events.

$\endgroup$
2
  • $\begingroup$ It is not a good idea to do variable selection and effect estimation in the same dataset, and OP clearly doesn't have enough data to properly split their sample, so they really should just rely on theory to determine confounders to adjust for. This still does not get at their question of how to adjust for the confounders. $\endgroup$
    – Noah
    Dec 14, 2021 at 1:14
  • $\begingroup$ So you are saying that I should not check on my dataset, if covariates are relatd to my endpoint (survival) and instead rely on what the literature says? $\endgroup$
    – Takanashi
    Dec 14, 2021 at 20:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.