Average Treatment Effect after Propensity Score Matching

Question

I used the "Matchit" package in R to perform propensity score matching on my data set and want to calculate the average treatment effect afterward. I use a caliper of 0.2 Std. deviations and the full method to avoid a 1:1 matching ratio (Purpose: Increase validity).

I used the following code:

x <- matchit(Treatment ~ x1 + x2 + ... xn,
                method = "full",
                replace = TRUE,
                distance= logit_PSM,
                caliper = 0.2,
                data = PSM_m1_clean
                )

Everything has worked fine, and I can match multiple control units with a single treatment unit. To calculate the average, I use logistic regression. Do I have to include weights in my regression, or can I ignore them? Because my results are significant without including weights.

reg <- lm(Outcome ~ Treatment, data = match.data(x))
#versus
reg <- lm(Outcome ~ Treatment, data = match.data(x), weights = weights)

Answer 1

A few things:

1. Setting replace = TRUE with full matching does nothing. See the documentation for full matching to see which arguments are allowed. The fact that you included this suggests you don't know exactly what full matching is, in which case you should read more about it on its documentation page or in the vignette on matching methods.

2. Setting a caliper doesn't necessarily increase validity. It just improves balance. But It can also induce other kinds of bias. You say you want to estimate the average treatment effect (ATE), but by default, the specification you used performs matching for the average treatment effect on the treated (ATT), not the ATE. Using a caliper also moves the estimated effect away from either estimand. So you need to decide which estimand is of interest to you and use that to inform how to perform the matching. I have an article about making that choice here.

3. You MUST include the matching weights. Otherwise it's like you didn't do matching at all. I wrote a whole vignette explaining exactly how to estimate treatment effects after matching. Please read it and follow the instructions. If your outcome is binary (you say you want to use logistic regression but your code uses lm(), which is for linear regression), you need to decide which contrast is of interest to you (e.g., risk difference, risk ratio, odds ratio, number needed to treat, etc.) and then use an appropriate method for estimating it. This is all described in the vignette. In addition, you must use a special standard error that accounts for matching; the usual standard errors are invalid. This is explained in the vignette.