# Large inconsistency between ATE estimates from geralized random forest, propensity score matching etc

I have a RCT data set with about 10,000+ observations and 50+ covariates, and I am trying to estimate the ATE and compare the estimates from a couple of models. The models I use are:

1. Geralized random forest (GRF): using the average_treatment_effect function
2. Propensity score matching (PSM): I estimate the propensity score function of being treated and match each control unit with the closest treated unit (using the Matchit R package)
3. Linear model in which the covariates are control variables

However, the point estimates are very different: GRF gives 0.010 with a standard error of 0.021; propensity score matching gives 0.224 with a standard error of 0.051; the linear model gives 0.136 with a standard error of 0.036. From my past experience, this inconsistency seems unusual - different methods often yields quite consistent estimates, but it is not the case here.

In general, why would there be huge differences between those estimates?

All of these methods estimate different estimands. The GRF estimates the average treatment effect (ATE). Propensity score matching estimates the average treatment effect in the treated (ATT). Linear regression estimates an unspecified estimand that corresponds to neither. One should not expect these estimates to coincide since they are estimating different quantities. If you want to target the same estimand with all three approaches, you first need to decide on which estimand you want to target, since variations of all three methods can target either the ATE or ATT. I discuss how to choose the estimand and which methods target different estimands in my paper here. (GRF is a version of g-computation, which I briefly mention in the paper.)

To match for the ATE, you can either use the Matching package or use matching methods in MatchIt that allow for estimation of the ATE, like full matching. To use regression to estimate the ATE, center all the covariates (including binary covariates) at their means, and regress the outcome on the treatment, the centered covariates, and the treatment-covariate interactions. The coefficient on treatment can then be interpreted as an estimate of the ATE.

To use regression to estimate the ATT, train a regression model of the outcome on the covariates using the control group, generate predictions from the treated group, and then take the average difference between the observed outcome for the treated and the predicted outcome for the treated. Use bootstrapping to get the standard error. (This is also a method of g-computation.)

You can also look into weighting methods, which can be used to target several different estimands. The WeightIt package makes this easy and uses similar syntax to MatchIt.