1
$\begingroup$

I am obtaining confidence intervals using different bootstrap methods. When comparing them, what is does it mean to use Monte Carlo variation to look at how much the upper and lower limits vary for each bootstrap?

$\endgroup$
  • $\begingroup$ It sounds like you are quoting something. Could you provide a reference or link to the source? $\endgroup$ – whuber Apr 5 '12 at 19:04
  • $\begingroup$ Its a homework question. I have to test the performance of bootstrapping methods for confidence intervals. The Monte Carlo variation was a suggestion on how to do this. $\endgroup$ – user10319 Apr 5 '12 at 19:16
  • $\begingroup$ The problem is that "Monte Carlo variation" could mean a few different things, such as a distribution of a resampling statistic or the variation due to the fact that many bootstrapped estimates are approximations made by means of Monte-Carlo sampling and are thereby subject to sampling variation. Some more context would be helpful. If it's homework for a course having a real live teacher, consider consulting them about the intended meaning. $\endgroup$ – whuber Apr 5 '12 at 19:18
  • $\begingroup$ Ok, I think it means that they are subject to variation. Is working out the variance the same as looking at how much the confidence limits vary for each simulation in the bootstrap? $\endgroup$ – user10319 Apr 5 '12 at 19:48
  • $\begingroup$ This is a well written paper on this topic: On the Assessment of Monte Carlo Error in Simulation-Based Statistical Analyses (Koehler, Brown, Haneuse) amstat.tandfonline.com/doi/abs/10.1198/… $\endgroup$ – boscovich Apr 5 '12 at 20:57
1
$\begingroup$

The confidence intervals that you estimate with a bootstrap will be subject to uncertainty.

There are different ways that one might estimate said uncertainty

One option is to do so analytically. Alternatively, one could use a Monte Carlo simulation to estimate the uncertainty.

Choice of methods probably depends on the type of bootstrap being used.

For example, analytic techniques seem a lot easier for a parametric residual bootstrap than a non-parametric residuals bootstrap or a paired bootstrap.

$\endgroup$
0
$\begingroup$

Richard is right. For example it is not just that the intervals vary with sample size but they are only asymptotically accurate. Also the higher order bootstraps such as bootstrap t, BCa and double bootstrap are usually more accurate than the lower order bootstraps such as the percentile method becaause they converge to their asymptotic confidence levels faster. But for some parameters such as for confidence intervals for variances even for moderate sample sizes the higher order bootstraps undercover and can for certain sample sizes even be less accurate than the lower order bootstraps. This can be determined by applying Monte Carlo to approximately calculate their coverage. This was shown by Chernick and LaBudde (2010) in the American journal of Mathematical and Management Science.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.