3
$\begingroup$

I am trying to to calculate bootstrap confidence interval on an index calculated from a vector of values, and if the index is significantly greater than 0 in R.

For example, the vector of length 6: (0,0, 100, 30, 200,6).
And I calculate the index with:

J = (var(vector)/mean(vector)^2) - (1/mean(vector))

I am trying to use a method of accelerated bootstrap from another paper that has done it in SAS, but I don't know what the R equivalent is? I dabbled with using boot.ci, but I wasn't sure how to specify it and if it was correct.

The bit from the paper I was referring to reads:

"We used the accelerated bootstrap (Dixon 2001, SAS) to estimate 95% confidence intervals for all aggregation indices and to test whether the parameter estimated by the index J differed significantly from 0 at alpha = 0.05"

$\endgroup$
4
$\begingroup$

First a warning... the Bootstrap (as with most statistical methods) is unlikely to be reliable with such a small sample size. I would exercise caution if $n=6$ is a standard sample size in your case.

Lets simulate some data

set.seed(42)
n <- 30 #Sample size
x <- round(runif(n, 0, 100))

Lets refer to your index as $\theta$ and the estimator you provide for it as $\hat\theta$, which can be computed as follows.

theta_hat <- var(x)/mean(x)^2 - 1/mean(x)

For this simulated data, I get $\hat\theta = 0.2104$ and (by cranking $n$ wayyyy up) we have (roughly) $\theta = 0.32$.

Obtain the Bootstrap distribution

The Bootstrap algorithm is fairly straightforward to code up on your own.

B <- 10000 #number of bootstrap resamples
theta_boot <- rep(NA, B)
for(i in 1:B){
  #Select a bootstrap sample
  xnew <- sample(x, length(x), replace=TRUE)
  #Estimate index
  theta_boot[i] <- var(xnew)/mean(xnew)^2 - 1/mean(xnew)
}

#Plot bootstrap distribution
hist(theta_boot, breaks=30, xlab='theta', main='Bootstrap distribution')
abline(v=0.32, lwd=2, col='orange')

The resulting distribution looks like this, where the vertical line represents the "true" value of the index $\theta$.

enter image description here

Confidence intervals using the (percentile) Bootstrap

At this point, getting a confidence interval is very straightforward. Suppose you want a $95\%$ CI (i.e. $\alpha = 0.05$). You are looking for the points $L$ and $U$ such that $2.5\%$ of the Bootstrap samples are below $L$ and above $U$.

Mathematically, this is equivalent to setting $$L = \hat F^{-1}(\alpha/2) \quad\quad\quad U = \hat F^{-1}(1-\alpha/2),$$ where $\hat F$ is the "Bootstrap CDF". In R, this can be done simply by typing

alpha <- 0.05
quantile(theta_boot, c(alpha/2, 1-alpha/2))

For this data, we get a $95\%$ CI of $(0.101, 0.355)$.

The Accelerated Bootstrap

Although the method of the previous section is a straightforward and natural way to obtain endpoints for a confidence interval, there are several alternatives which have been shown to perform better in a variety of settings. The Accelerated Bootstrap is one such method.

The endpoints to the CI in this approach are found by considering the function $$g(u) = \hat F^{-1}\left(\Phi\left(z_0 + \frac{z_0 + z_u}{1-a(z_0+z_u)}\right) \right)$$ and setting $L = g(\alpha/2)$ and $U=g(1-\alpha/2)$. There are a lot of new terms in this function which I will now describe.

  • $\Phi(z)$ represents the standard normal CDF.
  • $z_0 = \Phi^{-1}(\hat F(\hat\theta)).$
  • $z_u = \Phi^{-1}(u).$
  • $a$ is an "acceleration constant".

Estimation of the acceleration constant is the last remaining "challenge" and will be discussed in the next section. For now, let's fix the value $a=0.046$. The accelerate Bootstrap CI can now be computed in R as follows.

#Desired quantiles
u <- c(alpha/2, 1-alpha/2) 

#Compute constants
z0 <- qnorm(mean(theta_boot <= theta_hat))
zu <- qnorm(u)
a <- 0.046 

#Adjusted quantiles
u_adjusted <- pnorm(z0 + (z0+zu)/(1-a*(z0+zu))) 

#Accelerated Bootstrap CI
quantile(theta_boot, u_adjusted)

This gives a new $95\%$ CI of $(0.114, 0.383)$, which has effectively "shifted" the CI bounds in the direction of the true value for $\theta$. (Side note: when $a=0$, the accelerated Bootstrap is known as the bias correction Bootstrap).

The following figure shows the Bootstrap distribution again, with vertical lines representing the Confidence intervals for each case.

enter image description here

Estimating the acceleration constant

The acceleration constant can (in some cases) be calculated theoretically from the data by assuming a particular distribution for the data. Otherwise, a non-parametric approach can be used.

Efron (1987) shows that for univariate sampling distributions, the acceleration constant is reasonably well approximated by $$\hat a = \frac{1}{6}\frac{\sum_{i=1}^n I_i^3}{\left(\sum_{i=1}^nI_i^2\right)^{3/2}}$$ where $I_i$ denotes the influence of point $x_i$ on the estimation of $\theta$. Efron proposes approximating $I_i$ using the infinitesimal jackknife, but others have demonstrated that the finite-sample Jackknife is often sufficient. Thus, each $I_i$ can be approximated by $$I_i = (n-1)[\hat\theta - \hat\theta_{-i}]$$ where $\hat\theta_{-i}$ represents an estimate of $\theta$ (your index) after removing the $i^{th}$ data point.

I <- rep(NA, n)
for(i in 1:n){
   #Remove ith data point
   xnew <- x[-i]
   #Estimate theta
   theta_jack <- var(xnew)/mean(xnew)^2 - 1/mean(xnew)
   I[i] <- (n-1)*(theta_hat - theta_jack)
}
#Estimate a
a_hat <- (sum(I^3)/sum(I^2)^1.5)/6

This leads to the accleration constant estimate of $\hat a = 0.046$ that was used in the previous section.

$\endgroup$
  • $\begingroup$ The accelerated bootstrap "perform better in a variety of settings" than the percentile bootstrap. But, why? $\endgroup$ – dfrankow Nov 25 '19 at 16:21
  • 1
    $\begingroup$ @dfrankow -- great question. I've added a link to a blog-post giving more information on this. Essentially, resampling methods allow us to estimate the bias of an estimator (see the Jackknife estimator for a simple example). The accelerated bootstrap also accounts for (and adjusts for) the possible skewness of the Bootstrap distribution. $\endgroup$ – knrumsey Nov 25 '19 at 18:14
3
$\begingroup$

Since the question mentioned boot.ci, I thought I would try to replicate the results of @knrumsey with the boot package.

A couple of notes. I copied my general code for using boot.ci with a function from here (with the caveat that I am the author of the code).

The results are similar to those of @knrumsey.

I can't confirm that the 'perc' and 'bca' methods are the same as those used in the original answer.

set.seed(42)
n <- 30 #Sample size
x <- round(runif(n, 0, 100))

library(boot)

Function = function(input, index){
                    Input = input[index]
                    Result = var(Input)/mean(Input)^2 - 1/mean(Input)
                    return(Result)}

Boot = boot(x, Function, R=10000)

hist(Boot$t[,1])

boot.ci(Boot, conf = 0.95, type = "perc")

   ### BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
   ### Based on 10000 bootstrap replicates
   ###
   ### Intervals : 
   ### Level     Percentile     
   ### 95%   ( 0.1021,  0.3521 )  

boot.ci(Boot, conf = 0.95, type = "bca")

   ### BOOTSTRAP CONFIDENCE INTERVAL CALCULATIONS
   ### Based on 10000 bootstrap replicates
   ###
   ### Intervals : 
   ### Level       BCa          
   ### 95%   ( 0.1181,  0.3906 )  
$\endgroup$
  • $\begingroup$ +1, very cool! I would almost guarantee the methods ‘perc’ and ‘bca’ are the same as above. (BCa stands for bias correction with acceleration constant). The slight discrepancies can be do to MC error (even though you set the seed, the Bootstrap procedure might be slightly different). The other reason could be a slight difference in the estimation procedure of the acceleration constant. $\endgroup$ – knrumsey Nov 23 '19 at 5:10
  • $\begingroup$ Thanks. The documentation for the boot package cites "Chapter 5 of Davison and Hinkley (1997)" for the procedures for the confidence intervals. So, that's probably the definitive source for questions on the procedures there. The documentation describes the bca option as "adjusted bootstrap percentile" even though most places on the internet say it is "bias correction with acceleration" (or something similar). $\endgroup$ – Sal Mangiafico Nov 23 '19 at 15:28
  • $\begingroup$ See also this StackExchange link for a couple of additional references. $\endgroup$ – Sal Mangiafico Nov 23 '19 at 15:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.