# For propensity score matching on binary outcome variables, why can the effect of treatment only be reported as the ATT instead of the ATE?

"If the outcome is dichotomous (self-report of the presence or absence of depression), the effect of treatment can be estimated as the difference between the proportion of subjects experiencing the event in each of the two groups (treated vs. untreated) in the matched sample. With binary outcomes, the effect of treatment can also be described using the relative risk..."

It seems to me that the difference of the proportions of subjects experiencing the event under treatment and non-treatment refers to the estimator for the Average Treatment Effect on the Treated (ATT), instead of the Average Treatment Effect (ATE).

Is calculating the ATE impossible under a binary outcome?

• From the quoted excerpt I take away that for binary outcomes you can compute the treatment effect estimates from sample proportions. For dose response you clearly wouldn't look at proportions. Where do you take the ATT vs ATE argument from? – Andreas Dzemski Mar 4 '18 at 9:55
• You are ignoring outcome heterogeneity (e.g., non-collapsibility of odds ratios) and assuming that matching is a good idea. There are many negatives to matching as discussed in fharrell.com/doc/bbr.pdf which also contains a statistical analysis plan that uses regression adjustment for propensity score in addition to the most important pre-specified predictors. – Frank Harrell Mar 4 '18 at 14:24
• @FrankHarrell Can you elaborate on what you mean by outcome heterogeneity? Are you saying that looking at the relative risk is bad in principle? – user321627 Mar 9 '18 at 5:20
• No, I meant that adjusting for heterogeneity in treatment selection does not adjust for outcome heterogeneity. For a completely proper analysis you need to have the major predictive covariates in the model along with the propensity score. I have details about this in fharrell.com/doc/bbr.pdf . Ignoring this gives treatment effects too close to the null by making the model not fit. – Frank Harrell Mar 9 '18 at 13:28

I think that the best way to consider ATT/ATC/ATE estimates is by examining them within a counterfactual framework that is relevant to our research question. That way we can assess in a straight-forward manner what each difference refers at. We state the assumptions made (e.g. $E[Y^0|A = 1, X] = E[Y^0|A= 0, X] \approx$ if people in treatment group $A=1$ would do "as bad" as the control group $A=0$ if they were not treated, given information on covariates $X$) and then we calculate the difference we care for. Generally assessing "differences in the groups" can be potentially misinterpreted.