My team is conducting propensity score matching with 1:1 nearest neighbor replacement for a case-control healthcare study.

While we're obtaining match rates of 80-90% with good covariate balance, we have noticed a handful of treatment subjects are matched to 20+ controls.

Is it acceptable to manually trim the number of matched records so that, for example, no treatment subject is matched to > 5 controls? This would be done following matching by ranking the PS matches and keeping the top matched pairs.

Or is it better to incorporate matching limits into the matching procedure itself rather than ex post adjustments?

My understanding is there's a bias-variance tradeoff when matching with or without replacement. We prefer matching with replacement to obtain less biased estimates of treatment effects. Trimming members would slightly increase bias but (ideally) reduce variance of our treatment effect estimates.

  • 1
    $\begingroup$ I don't think there is a "one-size-fits-all" answer here. (+1 though cause I want to see what people think) Ultimately the "smaller sized" group will be the one driving your SEs. If by doing these "matching limits" you notice a "good" variance reduction, I would go for it. If pretty much the same results, meh... Just keep it simple and mention it in the text. $\endgroup$
    – usεr11852
    Commented Mar 12, 2023 at 22:36
  • 1
    $\begingroup$ Why does this need to be done after the fact instead of simply requesting 5:1 matching to start with? $\endgroup$
    – Noah
    Commented Mar 13, 2023 at 21:20
  • $\begingroup$ @Noah Normally that would work, except our data is in member-month format so that there are multiple records per member (one record per month & year). We're matching with proc psmatch in SAS, which treats each record as a separate subject. I think you said MatchIt in R allows limits by member ID when there are multiple records per member. At the moment we're stuck using SAS. It's not a serious issue, which is why I'm floating manual trimming. My intuition is as long as we settle on a matching & trimming strategy before estimating the treatment effect we're OK (ie, no fishing allowed). $\endgroup$
    – RobertF
    Commented Mar 13, 2023 at 22:27
  • 1
    $\begingroup$ Yup, that's allowed. Shame you can't use R because it does have that feature. You can do whatever you want before estimating the treatment effect as long as you don't involve the outcome. Note that there are special methods of estimating the treatment effect and standard error that might not be straightforward in SAS after manipulating the proc psmatch output. $\endgroup$
    – Noah
    Commented Mar 14, 2023 at 1:38
  • 1
    $\begingroup$ The argument is called unit.id. $\endgroup$
    – Noah
    Commented Mar 14, 2023 at 14:39


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.