Why does it make sense to perform a bootstrap procedure before calculating the confidence intervals? Will they be more precise? And if so, can anyone explain why?
You can calculate bootstrap confidence intervals for complex situations, i.e. properties ("statistics") that are not easily accessible analytically. I'm thinking of things like bootstrapping generalization error of a predictive model*.
In other words, bootstrapping may still be possible in situations where you have no good assumption which distribution to base your confidence intervals on.
The choice parametric (analytical confidence interval based on known distribution) vs. non-parametric bootstrap is a trade-off:
- good parametric statistics will be more precise. But they may be totally off if the assumptions are violated (i.e. the distribution you chose was not appropriate).
- bootstrap is less precise (for a given number of original cases) but does not rely on particular distribution assumptions, so there's less danger of getting that part wrong*.
How can I decide which number of replications is a good number for calculating confidence intervalls? 100? 1000? 10000?
@MartenBuuis already gave you some idea how to approach this question. Here's another, very pragmatic one:
- Bootstrap, say, with nboot = 100 replications.
- repeat this 10 times
- check variability of the bootstrap results.
- if the variation you observe over the repetitions of the bootstrapping calculation is acceptable for your application, fuse the 10x100 calculations and use the result of that nboot = 10x100 = 1000 replications.
If they are not sufficiently precise, fuse the 10x100 calculations, go back to step 1 and 2 with nboot = 1000 replications.
You get the idea.