4
$\begingroup$

I am reviewing a paper that uses propensity matching. I think i get how it works but i am an amateur at this and was looking for some help.

The paper has 150 in the treatment group and 650 in the control group. They do the matching and state that they found 102 matches. So they toss out the remaining 48 in the treatment group. But matching works if they have identified all the confounding variables and included them in constructing a propensity score. Now, say I strongly suspect there is another variable that should have been included but was not. Would it be valid to say that the reason they were having difficulty finding matches was because there were ignored variables that were causing variation. Or does that have nothing to do with it?

$\endgroup$

2 Answers 2

2
$\begingroup$

The basic intuition for the common support problem is that if you are going to estimate the counterfactual for a treated observation by someone matched to that person, then you need to find someone similar in terms of the propensity score in the counterfactual state.

Typically you will want to match on all variables that affect both participation and the outcome to satisfy the CIA. While it is possible that the authors left something out the PS mode, it is also possible that they matched on an instrument (a variables that affects participation, but not outcomes). This will will make a mess of the common support, leaving lots of unmatched treated observations. See Bhattacharya J and Vogt WB, “Do Instrumental Variables Belong in Propensity Scores?” International Journal of Statistics & Economics 9(A12) (2012). 

$\endgroup$
2
$\begingroup$

There are a few reasons why members of the treated group were dropped. One is that common support adjustment was used. There are several ways to do this; one is by using the convex hull, and another is to estimate propensity scores and then remove units outside the region of overlap. Another reason may be that calipers were used in matching (this is more likely). In this way, treated units that don't have a close enough control unit will be discarded. Closeness is typically assessed by using the distance between each unit's propensity scores.

Even if the propensity score was estimated perfectly, units may still be dropped because of the lack of overlap between the groups or the lack of of control units close to the treated units. So it has little to do with having the correct propensity score and more about the degree of similarity between the treated and control groups to start. Of course, measuring that similarity requires propensity scores, and well-estimated propensity scores (i.e., including all relevant variables) will do a better job at characterizing the true degree of similarity between the groups, but won't inherently lead to better matches. In fact, a poor propensity score model might lead to more matches than one would expect, because the degree to which units are similar is being mismeasured, and thus possibly overestimated.

$\endgroup$
2
  • $\begingroup$ Well what I am saying here is that the 48 dropped subjects had a propensity score that was calculated. Now the paper doesn't state why the 48 were dropped. I, personally agree with Noah that the groups were not entirely comparable to start because the (participation only) variables they used to construct the propensity score were not the relevant confounders. $\endgroup$
    – Ssb
    Sep 1, 2016 at 19:48
  • $\begingroup$ @Ssb: From the healthcare perspective, the purpose of propensity methods (intuitively) is to approximate randomization via adjustment or matching on factors that determine treatment selection. Confounders can be adjusted for after treatment selection (group assignment) is handled via either matching or inclusion of the propensity score as a regressor. The confounders can be and hopefully are a completely different set of variables than those contained in the propensity score. $\endgroup$
    – Todd D
    Sep 1, 2016 at 19:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.