I have a multinomial logit model with three outcome categories and a variety of (primarily binary) independent variables.
How would I go about bootstrapping the average treatment effect and its confidence interval of a binary variable?
Right now my procedure is as follows:
After running the full model as a multinomial logit I
- Sample with replacement to create temporary data of same length as original dataset.
- Reestimate the model using temporary data
- set set indep. variable to 0 in temporary data and predict probabilities of outcomes (using reestimated model)
- set set indep. variable to 1 in temporary data and predict probabilities of outcomes (using reestimated model)
- Take the mean of the average difference in predicted probabilities
- Save this mean difference in pred probabilities
- Repeat this process a lot of times
- Take mean of all the mean differences pred. probabilities for average treatment effect, Take 0.25 and 0.975 quantile for a 95% confidence interval around the mean.
The issue I am having right now is, that this seems to create very large confidence intervals. This has created a situation where a lot of the variables that are significant in the regression output no longer have an effect that is differentiable from 0.
- Is this bootstrap procedure sensible?
- If so, what explains the difference between the regression outputs and the bootstrap results?