I am conducting a retrospective case-control study, it is a clinical project. Initially I manually matched cases to controls of the same age and sex. It had been suggested that to have a less biased methid than manual matching, I should use propensity score matching. However, all I can see is how to use PSM as a method of statistical analysis, once data is collected.

My question is: is it possible to use PSM to identify corresponding controls, and then go and extract data for these controls . I have 60 cases and a pool of over 2000 controls to choose from. It would not be practical to extract data for all 2000 controls when only 60 will be used (1:1 match). I want to know, can i identify the 60 controls, and then go extract the data afterwards?

Thank you.

  • $\begingroup$ PSM is very inefficient and requires the data to already be extracted so that you can fit the propensity model. Please clarify. How many candidate variables for the PS are there? How were these variables chosen? What is the selection process you are trying to control for? $\endgroup$ Commented Jan 6 at 7:13
  • $\begingroup$ The actual matching variables I am hoping to use are 1)age 2)sex 3)anatomical location. There are many many clinical variables that I will then need to extract for the matched pairs (demographics, comorbidities, surgical and technical variables etc >30 at least). Essentially I have 60 cases and a pool of 2000 controls to select from. I initially manually matched the cases with a control with corresponding age, sex and anatomical location. I received feedback that this may introduced selection bias, and that PSM would reduced this bias. Thanks. $\endgroup$
    – user404553
    Commented Jan 6 at 7:18
  • 1
    $\begingroup$ You do not have nearly enough cases to get a reliable analysis that requires adjustment for a single variable, much less for multiple variables. Whatever you do don't compute P-values but rather (wide) confidence intervals for effects of interest. What makes you not use all 2000 controls? Are they expensive to extract? There is no gain in dropping controls, and you create a non-reproducible result if you sample controls differently from the way someone else might sample them. $\endgroup$ Commented Jan 6 at 7:29
  • $\begingroup$ The simple answer to your question is "no". I second @FrankHarrell's question about why not use all 2000 controls? Finally, in general, for a propensity score you use all the variables that might be relevant. $\endgroup$
    – Peter Flom
    Commented Jan 6 at 11:05
  • $\begingroup$ What is the hypothesis that you want to test via the case/control study? That might provide some guidance about an efficient way to proceed (even though the simple answer to your question is indeed "no"). Please provide that information by editing the question, as comments are easy to overlook and can be deleted. $\endgroup$
    – EdM
    Commented Jan 6 at 20:31

1 Answer 1


When there is a large imbalance between cases & controls in a retrospective dataset, there will likely be a large difference in confounders, or nuisance factors - things that are not being studied. Randomized clinical trials, however, employ inclusion and exclusion criteria to ensure that similar patients are enrolled in the study. In addition, randomization of subjects to the various arms attempts to reduce confounder differences between arms, but it has nevertheless still been observed in sample sizes $n<1500$. Since propensity matching attempts to extract data from a retrospectively collected dataset for which there was no inclusion/exclusion criteria applied, and no randomization, certain subjects in the retrospective dataset will need to be dropped. Which subjects? Subjects whose confounder values (medical histories, severity of disease, disease duration, repeated recurrences, etc.) are significantly different from cases (diseased). Why? Because the goal is to reduce heterogeneity across nuisance factors among subjects with which treatment efficacy and adverse events (safety) are being studied. You cannot "be all over the map" with variation in nuisance factors across subjects/arms -- since you want a highly controlled experiment. Strong heterogeneity in confounder influences across treatment (dose) arms diminishes the chance of observing efficacy. One of the worst scenarios for confounder heterogeneity results from studying subjects that are not de novo cases, i.e., are not newly diagnosed, but rather have been diseased for several years with extensive therapeutic histories. Subjects with long duration of disease and high number of repeat occurrences will have huge amounts of "excess baggage" which can render a trial to be futile (useless).

For propensity matching, I often first run a single logistic regression model with dependent variable values: y=1 if case, y=0 if control and independent (predictor) variables based on confounder or nuisance factors -- not being studied -- whose values are significantly different between cases & controls. Next, sort/get the predicted "logits" and their range for the cases, and then find an equal number of controls whose (sorted) logits are in the same range.

Then run the analysis of choice, such as the logrank test or Cox PH regression on survival for selected subjects with the new binary (0,1) treatment (exposure) variable. If there's a lot of controls whose predicted logit values are in the same range of predicted logit values of cases, then I just use resampling for the e.g. logrank test or Cox PH regression, randomly fetching controls with logits in the same range as cases and performing logrank test or Cox PH ~500 times. This is a "bootstrapped" analysis based on randomly selecting controls via if statements like

enter image description here

Note: cases will have the greatest predicted logit values since they are cases (y=1). Logits are used because logits are more normally-distributed when compared with y={0,1}.

There are limitations of propensity matching, especially when logits of most of the controls are nowhere near those of the cases, indicating large differences in confounder influences. Large differences in severity of disease will also cause this, as prior meds/therapies, histories, and comorbidities (all nuisance factors, confounders) will be very different between cases & controls. The logistic run mentioned uses confounders which are significantly different between cases & controls as input predictor. The logits are then predictors of "caseness."

There's no rationale for using propensity matching when logits for confounders are extremely different between cases and controls. As far as dropping subjects from an analysis, if age was not being studied in a clinical trial (nuisance factor, confounder), would you e.g. want to include controls of age 18-40 in a propensity-matched study of Alzheimer's disease? Answer: No. 18-40 year-old controls likely won't have Alzheimer's, will have fewer prior meds, negligible histories, fewer comorbidities, etc. Young, low-risk subjects enrolled in a drug trial for a novel molecule for Alzheimer's disease would not respond to treatment and would contribute nothing to efficacy. Propensity matching would be able to identify and drop such patients.

Most academic clinicians who perform randomized clinical trials know the rule-of-thumb: "A clinical trial for a high-risk treatment will fail if low-risk patients are enrolled." Low-risk patients will have confounders or nuisance factors not being studied (e.g. histories, prior meds, comorbities) that are significantly different from high-risk patients. Propensity matching attempts to find and eliminate (drop) patients whose confounder values are significantly different from those of the cases.

  • $\begingroup$ FYI - you need to set both userec=0 and usecntl=0 in the first two rows inside the boostrap loop. $\endgroup$
    – wjktrs
    Commented Jan 6 at 20:20
  • 1
    $\begingroup$ There is no rationale whatsoever for @wjktrs's approach. Even extreme imbalance is not harmful. Removal of valid observations is not scientific. Sampling of abundant controls is only useful when there is a future per-person cost to sampling and you want to save money. For example if the sampling involves thawing a blood sample from the freezer and it costs $200 to analyze each blood sample. $\endgroup$ Commented Jan 7 at 7:52
  • $\begingroup$ Thanks for the comment - my answer focused on a large confounder imbalance, which would likely obtain from a large sample size difference. Would young, 18-40 year-old controls/subjects be valid observations for a study of Alzheimer's disease. Clinicians who use propensity matching often switch the original disease focus of a dataset and extract a different (treatment) factor as well as a different outcome based on measured diagnostic data - essentially turning trial data or case/cntl data on its head. $\endgroup$
    – wjktrs
    Commented Jan 7 at 10:00
  • 2
    $\begingroup$ Propensity matching has serious problems, and at its best is inefficient (yields higher standard errors, lower power). You can study Alzheimer's disease in an unnecessarily wide age distribution without problems as long as you adjust (smoothly and nonlinearly) for age. Adjusting for age through a smooth spline function in a regression context is so easy to do and is more efficient. $\endgroup$ Commented Jan 7 at 13:53
  • 1
    $\begingroup$ Agree - we used to say to clinicians: just run logistic regression and adjust for nuisance factors in the model. In fact, that's the common recommendation -- not to perform propensity matching. I never trust data anyhow, so I always bootstrap or resample to gain a handle on uncertainty. If a study is replicated 10,000 times, the statistics will form an uncertainty distribution. $\endgroup$
    – wjktrs
    Commented Jan 7 at 19:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.