# Bootstrapping Average Treatment Effects and Confidence Intervals for multinomial logit models

I have a multinomial logit model with three outcome categories and a variety of (primarily binary) independent variables.

How would I go about bootstrapping the average treatment effect and its confidence interval of a binary variable?

Right now my procedure is as follows:

After running the full model as a multinomial logit I

1. Sample with replacement to create temporary data of same length as original dataset.
2. Reestimate the model using temporary data
3. set set indep. variable to 0 in temporary data and predict probabilities of outcomes (using reestimated model)
4. set set indep. variable to 1 in temporary data and predict probabilities of outcomes (using reestimated model)
5. Take the mean of the average difference in predicted probabilities
6. Save this mean difference in pred probabilities
7. Repeat this process a lot of times
8. Take mean of all the mean differences pred. probabilities for average treatment effect, Take 0.25 and 0.975 quantile for a 95% confidence interval around the mean.

The issue I am having right now is, that this seems to create very large confidence intervals. This has created a situation where a lot of the variables that are significant in the regression output no longer have an effect that is differentiable from 0.

Two questions:

1. Is this bootstrap procedure sensible?
2. If so, what explains the difference between the regression outputs and the bootstrap results?

2. It could be that your logit model overfit/is not close to the "true model", therefore its CIs are too small. Especially, a big thing to look out for is positivity issues. Suppose your binary covariates are $W_1, W_2, W_3$ respectively, and for $(W_1, W_2, W_3) = (0,1,0)$ you only have 2 observations in this category. They're highly influential, being the only points in this category, but would likely be excluded in the bootstrapped data. This may change your estimated ATE by a lot.