I am obtaining confidence intervals using different bootstrap methods. When comparing them, what is does it mean to use Monte Carlo variation to look at how much the upper and lower limits vary for each bootstrap?
$\begingroup$
$\endgroup$
7
-
$\begingroup$ It sounds like you are quoting something. Could you provide a reference or link to the source? $\endgroup$– whuber ♦Apr 5, 2012 at 19:04
-
$\begingroup$ Its a homework question. I have to test the performance of bootstrapping methods for confidence intervals. The Monte Carlo variation was a suggestion on how to do this. $\endgroup$– user10319Apr 5, 2012 at 19:16
-
$\begingroup$ The problem is that "Monte Carlo variation" could mean a few different things, such as a distribution of a resampling statistic or the variation due to the fact that many bootstrapped estimates are approximations made by means of Monte-Carlo sampling and are thereby subject to sampling variation. Some more context would be helpful. If it's homework for a course having a real live teacher, consider consulting them about the intended meaning. $\endgroup$– whuber ♦Apr 5, 2012 at 19:18
-
$\begingroup$ Ok, I think it means that they are subject to variation. Is working out the variance the same as looking at how much the confidence limits vary for each simulation in the bootstrap? $\endgroup$– user10319Apr 5, 2012 at 19:48
-
$\begingroup$ This is a well written paper on this topic: On the Assessment of Monte Carlo Error in Simulation-Based Statistical Analyses (Koehler, Brown, Haneuse) amstat.tandfonline.com/doi/abs/10.1198/… $\endgroup$– boscovichApr 5, 2012 at 20:57
|
Show 2 more comments
2 Answers
$\begingroup$
$\endgroup$
The confidence intervals that you estimate with a bootstrap will be subject to uncertainty.
There are different ways that one might estimate said uncertainty
One option is to do so analytically. Alternatively, one could use a Monte Carlo simulation to estimate the uncertainty.
Choice of methods probably depends on the type of bootstrap being used.
For example, analytic techniques seem a lot easier for a parametric residual bootstrap than a non-parametric residuals bootstrap or a paired bootstrap.
$\begingroup$
$\endgroup$
Richard is right. For example it is not just that the intervals vary with sample size but they are only asymptotically accurate. Also the higher order bootstraps such as bootstrap t, BCa and double bootstrap are usually more accurate than the lower order bootstraps such as the percentile method becaause they converge to their asymptotic confidence levels faster. But for some parameters such as for confidence intervals for variances even for moderate sample sizes the higher order bootstraps undercover and can for certain sample sizes even be less accurate than the lower order bootstraps. This can be determined by applying Monte Carlo to approximately calculate their coverage. This was shown by Chernick and LaBudde (2010) in the American journal of Mathematical and Management Science.
answered Jun 5, 2012 at 2:59