Understanding Propensity Score Matching

Question

I am trying to better understand the motivations and the applications behind Propensity Score Matching.

I read the following that explains the motivations behind Propensity Score Matching:

• Suppose you are want to conduct a study on the effectiveness of a certain pharmaceutical drug on Diabetes patients. However, all you have is "observational data" : you only have data on patients with diabetes that are currently taking this drug and patients who are not taking that drug.

• However, there might exist certain biases and reasons that explain which patients are more likely to be taking this drug. For example, this drug on average is prescribed to younger patients with Diabetes compared to older patients. Or this drug might not be covered by common insurance plans - therefore, patients earning over a certain income level are more likely to take this drug compared to patients earning under a certain income level. However, all this might be unknown to you at the time.

• Based on information within the data (i.e. covariates associated with each patient), Propensity Score Matching attempts to find "similar" patients taking this drug and their "counterparts" who are not taking the drug. Propensity Score Matching tries to predict the probability of the drug being prescribed to a individual patient within an observational study.

• As I understand, Propensity Score Matching will allow to "trim" the population so that there is "little difference" (i.e. variation) between the patients taking the drug vs the patients not taking the drugs. In other words, Propensity Score Matching tries to mitigate the problem of "comparing apples to oranges". Ideally, this would serve to reduce bias in the results of your study, ensuring that the groups of patients analyzed were as similar as possible.

The following picture illustrates Propensity Score Matching:

I was looking at the algorithm details of Propensity Score Matching - in short, it seems to contain 3 steps:

1) Run a Logistic Regression model to estimate the probability of a patient receiving the treatment vs not receiving the treatment.

2) Based on these Propensity Score Estimates, create pairs of patients from the treatment/non-treatment groups using some predefined method (e.g. KNN). (I am not sure why Propensity Score needs to be calculated for the matching - does the Propensity Score simply serve as a "diagnostic check" to ensure "balance" between both groups?)

3) Run your statistical models and analysis on these groups

My Question: In Step 1), is it necessary to use a Logistic Regression Model to estimate the Propensity Scores? In theory, could a Random Forest model be used to estimate these Propensity Scores - or is the interpretability of the Logistic Regression Model (i.e. effect and confidence intervals of each variable on the model) necessary for Propensity Score Estimation?

Thanks!

References:

• https://en.wikipedia.org/wiki/Propensity_score_matching

Answer 1

What you described in the text before the images is just "matching". Propensity score matching is one type of matching that uses the difference between two units' propensity scores as the distance between them. There are several other popular ways of computing the distance between them, some of which do not involve the propensity score at all (e.g., Mahalanobis distance matching). The reason the propensity score difference is popular as a distance measure is that there is some theoretical support for its use (described in the original Rosenbaum and Rubin (1983) paper introducing the method) and it tends to work well in practice at creating balanced groups.

Propensity scores can be estimated using any method that produces predicted probabilities, including logistic regression and machine learning methods such as random forests. Other popular methods include gradient boosted trees, lasso logistic regression, and Bayesian additive regression trees. The way to choose which method to use is to see which one produces the best balance (measured broadly) in the matched dataset. See my answer here.

I highly encourage you to read some introductory papers on propensity score methods. I think Austin (2011) is an excellent start. If you're interested specifically in matching (as there are other ways to use the propensity score), Stuart (2010) is another excellent introduction. I also encourage you to read the MatchIt vignettes to see how matching can be done in practice. I've also written extensively about propensity scores and matching so you are welcome to peruse my contributions to the tag.

Answer 2

My opinion is that adjusting by the PS is not a good ide. It is true that a circumstance under which a linear estimate will not change in expectation is if a covariate that is fitted is balanced between the two groups. However

1. It is not the only circumstance under which the estimate will not change in expectation. The other is that the covariate is not predictive.
2. It is not true that inference will not change. The standard error will be expected to change even if the covariate is balanced if it is predictive. For example the estimate from a matched pair design will be the same whether or not you fit 'pair' in the model but the inference will be quite different.

Putting these two together, it seems logical to choose covariates on which to condition not because they are predictive of assignment (PS) but because they are predictive of outcome (ANCOVA).

Also I would be wary of following the causal inference teaching on this. Some who are involved in this don't seem to care about standard errors. In my opinion this is a fundamental mistake.The importance of knowing how much you don't know See

1. Senn SJ, Graf E, Caputo A. Stratification for the propensity score compared with linear regression techniques to assess the effect of treatment on exposure. Research. Statistics in Medicine. Dec 3 2007;26(30):5529-5544.