6
$\begingroup$

I would like to use a greedy nearest neighbour method to do propensity score matching. Though I've little experience here, it seems that the distance measure used is generally a propensity score generated from a logistic regression. My question is: why logistic regression? Why not a random forest, SVM or another method? Is there some logic to suggest this would be unfruitful?

My plan is currently to use the MatchIt package in R and input my own distance measure calculated off the back of a random forest (you can input your own propensity score into the argument distance of the matchit function).

I'm deterred by the fact that modern propensity scoring packages such as PSAboot don't have built in facility to do this for nearest neighbour methods. They do use party and rpart but only to match using strata (additionally they only use single trees).

I'm intrigued that methods with (typically) greater predictive power than logistic regression do not appear to be harnessed to create a 'better' propensity score for one-to-one matching. Is there someone out there that can shed some light on this?

Linked question: Propensity Score

$\endgroup$
2
  • 1
    $\begingroup$ Greg Ridgeway has done a bit of work on using gradient boosting and then using those predictions for propensity score models. So your idea is not totally without precedent. $\endgroup$
    – Andy W
    Commented Aug 29, 2014 at 17:24
  • $\begingroup$ Thanks @AndyW. His 2004 paper is v interesting. It even gives some R code at the bottom. $\endgroup$ Commented Aug 30, 2014 at 17:23

1 Answer 1

2
$\begingroup$

Logistic regression is mostly likely used because of historic convenience, well studied convergence properties, relative data-frugality in comparison with other ML learners as well as being readily available pretty much everywhere. Also the resulting probabilities are usually "well-calibrated" out-of-the-box and this is helpful as it does not lead to under-/over-estimation of the probability to receive treatment as well as makes the occurrence of "extreme probabilities" (near 0 or 1) less likely. GBMs specifically, do not give very well-calibrated probabilities out of the box, I provided relevant material and commentary in this CV.SE tread on: Biased prediction (overestimation) for xgboost. Similarly simply using a tree would not be strongly advisable as it would lead to discontinuities and non-strictly monotonic probabilities that could mess up ordering because of the ties. Finally SVMs are a bit of red-herring as strictly speaking they do not provide probabilities natively but we need to use Platt scaling to get a similar output. This brings us to the last point: we can always post-process our output to make it better calibrated. The success of that step will be crucial but to avoid getting it "very wrong" logistic regression presents a safe bet. I recently read A tutorial on calibration measurements and calibration models for clinical prediction models by Huang et al. and I found it very informative if you want to explore that point further.

$\endgroup$
3
  • $\begingroup$ Are you aware of any discussion regarding the relevance of calibrated probabilities in the performance of propensity scores and derived weights? $\endgroup$
    – Kuku
    Commented Mar 6, 2023 at 12:26
  • 1
    $\begingroup$ No I am not. Maybe you want to see though "Balancing vs modeling approaches to weighting in practice" by Chattopadhyay et al. (2020) as it compares different approaches in propensity scores estimation. $\endgroup$
    – usεr11852
    Commented Mar 6, 2023 at 13:09
  • $\begingroup$ Will have a look for sure, thanks! Was asking as you say " [well-calibrated probabilities are] helpful as it does not lead to under-/over-estimation of the probability to receive treatment [...] ", which intuitively makes sense, as I would assume that propensity scores that will be used in individual-specific weights must be calibrated. However, the wide use of boosting algorithms and their relatively good performance in causal inference (e.g. GBM and BART) makes me doubt whether my intuition is correct, yet I cannot find any place where such assumption is made explicit and discussed. $\endgroup$
    – Kuku
    Commented Mar 6, 2023 at 13:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.