6
$\begingroup$

I have reviewed a number of questions, but so far there is no clear answer on exactly which algorithms/procedures are implemented in boot.ci for R. For instance, 292619 mentions the types of intervals given but there it is in the context of selecting one of them, not defining the method.

The intervals are listed as:

  • Normal
  • Basic
  • Studentized (Likely the bootstrap t, as in this paper, p. 1150)
  • Percentile
  • BCa (This one "appears" clearly defined!)

Unfortunately, these terms are not unambiguous as Michael Chernick points out, it is not clear if the percentile method is Efron's or Hall's (see comments on question). Additionally there is no clear linking from these methods to the sources. Nor do the docs is R clarify these points.

For instance, is the "normal" in this function the normal interval estimated by using the variance of the bootstrap distribution in the construction of the usual normal-theory CI? Or is that one the "basic" bootstrap? See here, page 216, if you can access this.

Is there a definitive source that lists the methods implemented and where the algorithms come from? Preferably a source that is available without trying to get a textbook?

$\endgroup$
2
  • 1
    $\begingroup$ The documentation for boot.ci lists several references. One is a book. cran.r-project.org/web/packages/boot/boot.pdf $\endgroup$ Commented Mar 2, 2018 at 18:24
  • $\begingroup$ @Great38 -- That link only goes the the "man page" for the package and does not give any additional information. Also, there is no clear link to the citations given in the man page and the names of the methods. The Davison and Hinkley book is unavailable to me. :-( And reading the paper from Statistical Science there are multiple methods with the same/similar names mentioned, so linking them to the boot.ci output is not possible (for me anyway). $\endgroup$ Commented Mar 2, 2018 at 18:34

2 Answers 2

5
$\begingroup$

This question was asked some time ago but remains relevant, and the only other answer is incomplete & partially incorrect.

To highlight the issue, the same intervals are called by different names across reference works (see full list below and this SAS note):

Name in boot_ci Names in literature
Percentile S&T: bootstrap percentile
E&T: percentile
Hall: other percentile
Hjorth: Efron's backwards percentile
Basic S&T: hybrid
Hall: percentile
Hjorth: simple
(not named in E&T)
Studentized S&T, E&T: bootstrap-t
Hall: percentile-t
Hjorth: studentized

The 'normal' and 'BCa' intervals are more consistently named. Let's go over each of them, sticking to the boot names but focusing only on the calculations and not on the assumptions and trade-offs that come with each of them.

All of these assume you want to calculate a two-sided $1-\alpha$ interval $(\mathrm{L}, \mathrm{U})$ around a sample statistic $t_0$ on which you have bootstrapped a distribution of $n$ resamples with the statistics $t$ (not to be confused with the studentized interval, this is boot's parameter naming).

Percentile

As simple as it gets, the $(\alpha/2, 1-\alpha/2)$ sample quantiles of the bootstrap distribution.

Basic

boot uses some pretty complex code in its calculation, but this can be drawn directly from the sample quantiles: if the percentile interval is $(\mathrm{L}, \mathrm{U})$ then the basic one is $(2t_0-\mathrm{U}, 2t_0-\mathrm{L})$. Note that this does not use any approximation, normal or otherwise, it's simply inverting the quantile distribution around the original point estimate.

Normal

Relies on a normal approximation, where the bootstrap distribution's standard deviation $\sigma$ is used to return the interval $t_0 \pm \sigma\times\Phi^{-1}(1-\alpha/2)$. boot adds an additional bias correction, centering the interval around $2t_0-\bar t$ instead of just $t_0$.

Studentized

Besides the bootstrapped sample statistics $t$ this interval also requires a within-sample estimation of their variances $v$. These are turned into within-sample Z-scores $z=(t-t_0)/\sqrt v$. The sample quantiles of those $z$ are then used instead of the standard normal quantiles $\Phi^{-1}$ in the formula of the normal interval above, as well as the original sample's $v_0$ instead of the bootstrapped $\sigma$.

Bias-Corrected (BC)

This is not directly exposed within boot, but the below BCa interval builds on BC so it's useful to discuss the calculation separately. First the median bias is calculated as $z_0=\Phi^{-1}\Big(\frac{I(t<t_0)+I(t=t_0)/2}{n}\Big)$ where $I$ is the indicator function. Then, $\alpha$ is recalculated to use the following values rather than the specified ones:

$$ \alpha_\mathrm{L}=\Phi(2z_0+\Phi^{-1}(\alpha/2)) $$ $$ \alpha_\mathrm{U}=\Phi(2z_0+\Phi^{-1}(1-\alpha/2)) $$

The BC interval is then the $(\alpha_\mathrm{L}, \alpha_\mathrm{U})$ sample quantiles of the bootstrap distribution.

Bias-Corrected and Accelerated (BCa)

This interval requires an estimate of the acceleration $a$. See here for a bit more details on how boot derives this, note that it is a property of your original sample and not of the bootstrap. $a$ is used as an extra adjustment to the BC interval:

$$ \alpha_\mathrm{L}=\Phi(z_0+\frac{z_\mathrm{L}}{1-z_\mathrm{L}a}),\ \mathrm{where}\ z_\mathrm{L} = z_0+\Phi^{-1}(\alpha/2) $$ $$ \alpha_\mathrm{U}=\Phi(z_0+\frac{z_\mathrm{U}}{1-z_\mathrm{U}a}),\ \mathrm{where}\ z_\mathrm{U} = z_0+\Phi^{-1}(1-\alpha/2) $$

The BCa interval is again the $(\alpha_\mathrm{L}, \alpha_\mathrm{U})$ sample quantiles of the bootstrap distribution.

References

D&H: Davison, A. and Hinkley, D. (1997) Bootstrap Methods and Their Application, Cambridge University Press.
E&T: Efron, B. and Tibshirani, R.J. (1993), An Introduction to the Bootstrap, Chapman & Hall.
Hall: Hall, P. (1992), The Bootstrap and Edgeworth Expansion, Springer-Verlag.
Hjort: Hjorth, J. (1994), Computer Intensive Statistical Methods, Chapman & Hall.
S&T: Shao, J. and Tu, D. (1995), The Jackknife and Bootstrap, Springer-Verlag.

$\endgroup$
1
$\begingroup$

From their help and code they have:

Normal: Using the normal approximation to a statistic, calculate equi-tailed two-sided confidence intervals.

Basic: Normal approximation method for confidence intervals. This can be used with or without a bootstrap object. If a bootstrap object is given then the intervals are bias corrected and the bootstrap variance estimate can be used if none is supplied.

Studentized: Studentized version of the basic bootstrap confidence interval.

Percentile: Bootstrap Percentile Confidence Interval Method. (they normalize it by Interpolation on the normal quantile scale. For a non-integer order statistic this function interpolates between the surrounding order statistics using the normal quantile scale. See equation 5.8 of Davison and Hinkley (1997)

I know that it's not much but I hope it helps a tiny bit.

$\endgroup$
2
  • $\begingroup$ Do you interpret the "normal" as the interval constructed by estimating the standard error of the statistic from the bootstrap distribution? Or do you think it uses some bias correction? When I tested it it looked like the latter, but I was not sure. I agree that this is what the manual says, but unfortunately this is still vague. However your answer did lead me to the source code, and I am hoping that will help. Thanks for the input! $\endgroup$ Commented Mar 2, 2018 at 20:26
  • $\begingroup$ @Doctorambient if the bootstrap object is provided, then it uses bias correction. At least according to their code. Don't forget to upvote if the comment helped :). $\endgroup$ Commented Mar 3, 2018 at 19:06

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.