2
$\begingroup$

I have been reading around the literature and have been trying to work out the correct way (or most accurate way) to calculate a 68.3% confidence interval using bootstrapping for my particular data sample, but wasn't 100% clear so far.

I have a bootstrapped parameter distribution that is non-normal, and has a definite skew to the right (see attached image). It has been suggested to me to simply determine the confidence interval (which will be asymmetric around the mean in this case) by removing N * 0.5 * (1-0.683) of the N bootstrapping results from left and right and then taking these end points as the 15.85% and 84.15% quantiles. From reading around, it seems that for skewed and/or biased bootstrap parameter distributions (as is the case here) I should instead use the BCa bootstrap interval to determine the confidence intervals, as this will provide more accurate intervals with better coverage for this particular situation than the nominal method described above (which I think assumes normality of the parameter distribution?)

Is this the correct interpretation, and if so could someone please explain to me why?

Thanks in advance.

bootstraped parameter distribution

Improve this question
$\endgroup$

1 Answer 1

Reset to default
3
$\begingroup$

If you can assume normality, which means that the distribution is determined by $\mu$ and $\sigma$, it is better to estimate these values (e.g. by means of the bootstrap) and base the confidence intervals on these estimators, i.e. $\mu\pm z_{1-\alpha/2}\sigma$).

If you cannot assume normality, your approach of simply taking the percentiles of the bootstraped computations of the observable is perfectly valid and known as the (non-parametric) percentile bootstrap interval. This method does not assume normality of the parameter distribution, but for unsymmetric distributions, there are examples where this confidence interval does not have a good coverage probability for smaller $n$. The $BC_a$ bootsrap interval ("bias corrected accelerated") compensates for this and typically has better coverage probability in such cases.

Out of curiosity, I had made Monte Carlo simulations to compare the different (non-parametric) bootstrap intervals, and the $BC_a$ bootstrap interval indeed had much faster convergence to the nominal coverage probability (see figures 7(a) and 8(a)):

Dalitz: "Construction of confidence intervals." Technical Report No. 2017-01, pp. 15-28, Hochschule Niederrhein, Fachbereich Elektrotechnik & Informatik, 2017

Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks a lot for the clarification! When you mention for particular cases with smaller n, do you suggest that the percentile bootstrap interval will converge to the same coverage probability as the BCa bootstrap interval in the limit of a large number of bootstraps, for these cases? $\endgroup$
    – Pronitron
    Oct 21, 2020 at 11:24
  • $\begingroup$ Not in the limit of a large number of bootstraps, but in the limit of a large number of the original data samples. When you look at the two figures 7a and 8a in the mentioned report, n is the number of original samples and the coverage probability eventually approaches the nominal 0.95 for all confidence interval. $\endgroup$
    – cdalitz
    Oct 21, 2020 at 12:16
  • $\begingroup$ Apologies I misinterpreted n. Right, interesting. My particular case has a small sample size (n~70), so this could well be a factor in this case. Thanks a lot, very helpful! $\endgroup$
    – Pronitron
    Oct 21, 2020 at 12:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.