1
$\begingroup$

I have a dataset with death outcome and around 25 independent variables.

I am planning to use logistic regression with PSM to understand the effect of treatment (a specific drug) which is one of the input variables on the outcome variable.

I am using PSM for the first time and have been following tutorials here,here.

Are the below called as different propensity score matching methods?

PS matching
PS weighting
PS stratification
PS as covariate

Are the above the different PS matching methods? If they are not, can you please let me know what are the different Propensity score matching methods?

Why is it said that we may/will get different pairs of data when we use different PSM methods? For ex, matching method 1 might give us 300 pairs of data (out of 2000 records in a dataset), another method gives only 100 pairs of data (out of 2000 records in a dataset). Why is this happening?

How will this impact when we compute balance measures such as SMD? Because the findings of SMD from each method may provide us different conclusion. Is there anything that we should do to address this?

Can someone help me with this in ordinary laymen terms and link to step-step tutorial if any please?

$\endgroup$
4
$\begingroup$

The word "matching" can sometimes be vague. Stuart (2010) defines matching "broadly to be any method that aims to equate (or "balance") the distribution of covariates in the treated and control groups". This would include PS matching, PS stratification, and PS weighting, but not PS as a covariate. Ho, Imai, King, and Stuart (2007) consider matching as nonparametric preprocessing to reduce model dependence. This would include PS matching and PS stratification, but not PS weighting or PS as a covariate. More specific definitions of matching refer to matching as subsetting (and optionally, but typically, pairing) the treated and control groups in order to equate the distribution of covariates in the treated and control groups, in which case only PS matching fits this definition. Don't get too hung up on the definition of "matching"; just try to be consistent and clear when describing what you are doing.

The four methods you mentioned all involve the propensity score, but that doesn't make them all matching methods. PS as a covariate is essentially a regression method just like covariate adjsustment but summarizing all the covariates into a single value. This method is almost always used improperly and has poor statistical performance when used improperly, so I recommend you do not use it if you are just starting out.

PS matching methods offer flexibility with the number of units retained. It's also important to know that the number of units retained is not the sole measure of the precision of the sample; for example, full matching retains all units but that doesn't mean no precision is lost when using it. Different ways of performing the matching can yield different numbers of remaining units. The simplest case is 1:1 nearest neighbor matching without replacement and without a caliper. This involves matching one control unit to each treated unit, so the number of remaining units should be twice the original size of the treated group. You can also add a caliper; this involves disallowing any pairs that are farther apart than some threshold set by the user. Any units without a close enough match will be discarded, so using a caliper will result in fewer units remaining. Matching with replacement allows each control unit to be matched to multiple treated units. If one control unit is a good match for many treated units, then fewer control units need to be used to find matches for all treated units, resulting in fewer units remaining. K:1 matching involves matching more than one control unit to each treated unit; this obviously increases the number of units remaining.

Different matching methods will produce different balance statistics. The whole point of matching is to try many matching methods and find the one which achieves the best balance in your dataset without discarding too many units. It is critical that you do not involve the outcome variable in this process.

The articles on the MatchIt website attempt to provide a tutorial for the use of matching, as well as descriptions of many matching methods, best practices in balance assessment, and effect estimation after matching. They include many references, which you should read before proceeding.

One thing I want to stress is that propensity score methods are advanced statistical methods that require highly specialized training to use correctly. Every single systematic review of the use of propensity score methods in published research indicates that many researchers are using the methods incorrectly. Tutorials may help you understand the basics of using the methods in the most ideal cases, but the most basic uses of these methods often yield the worst performance. I highly recommend you consult or collaborate with a statistician trained in the use of these methods before proceeding on your own.

$\endgroup$
4
  • $\begingroup$ Thanks a lot for the detailed response. upvoted. yes, agree that it's better to work with a statistician to apply the method correctly (esp causal inference is always tricky). May I check an additional thing with you? so our objective of matching is like optimization/feature engineering in ML where we try different methods and pick the best method (ex: Nearest Neighbours with calipers) which produces retains a decent amount of matched sample and produces best metric (in this case, SDM) and reject all other methods (ex: optimal matching, greedy matching and Nearest neighbours without calipers) $\endgroup$ – The Great Apr 5 at 6:13
  • $\begingroup$ Based on your response, it feels like only methods such as optimal matching, NN matching, NN with calipers matching seem to better fit the definition of matching (based on my english knowledge).. How would you differentiate the above 3 matching methods with PS matching, PS stratification, and PS weighting? $\endgroup$ – The Great Apr 5 at 6:17
  • $\begingroup$ I recommend reading Austin (2011), which explains what each of these is. PS matching is any kind of matching where the distance metric is the propensity score difference. PS weighting is computing weights as a function of the propensity score to weight the sample. PS stratification is splitting up the sample into bins based on the propensity score. They are three different methods with the same goal: balance the covariates, and they all involve the propensity score (though there are variations on all of them that don't use the propensity score). $\endgroup$ – Noah Apr 5 at 8:24
  • $\begingroup$ Hi @Noah, if you have time, would you be able to help me with this? stats.stackexchange.com/questions/518660/… $\endgroup$ – The Great Apr 8 at 4:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.