Firstly, after taking the time to digest the MIT chapter and the comments, the most important thing to note is that what MIT calls empirical bootstrap and percentile bootstrap differ - The empirical bootstrap and the percentile bootstrap will be different in that what they call the empirical bootstrap will be the interval $[\bar{x*}-\delta_{.1},\bar{x*}-\delta_{.9}]$ whereas
$$[\bar{x*}-\delta_{.1}, \bar{x*}-\delta_{.9}]$$
whereas the percentile bootstrap will have the confidence interval $[\bar{x*}-\delta_{.9},\bar{x*}-\delta_{.1}]$.
$$[\bar{x*}-\delta_{.9},\bar{x*}-\delta_{.1}].$$
I would further argue that as per Efron-Hastie the percentile bootstrap is more canonical. The key to what MIT calls the empirical bootstrap is to look at the distribution of $\delta = \bar{x} - \mu$ . But why $\bar{x} - \mu$, why not $\mu-\bar{x}$. Just as reasonable. Further, the delta's for the second set is the defiled percentile bootstrap !. Efron uses the percentile and I think that the distribution of the actual means should be most fundamental. I would add that in addition to the Efron and Hastie and the 1979 paper of Efron mentioned in another answer, Efron wrote a book on the bootstrap in 1982. In all 3 sources there are mentions of percentile bootstrap, but I find no mention of what the MIT people call the empirical bootstrap. In addition, I'm pretty sure that they calculate the percentile bootstrap incorrectly. Below is an R notebook I wrote.
Hide orig.boot = c(30, 37, 36, 43, 42, 43, 43, 46, 41, 42) boot = read.table(file = "boot.txt") means = as.numeric(lapply(boot,mean)) # lapply creates lists, not vectors. I use it ALWAYS for data frames. mu = mean(orig.boot) del = sort(means - mu) # the differences mu means del And
orig.boot = c(30, 37, 36, 43, 42, 43, 43, 46, 41, 42)
boot = read.table(file = "boot.txt")
means = as.numeric(lapply(boot,mean))
# lapply creates lists, not vectors. I use it ALWAYS for data frames.
mu = mean(orig.boot)
del = sort(means - mu) # the differences
mu
means
del
And further
Hide mu - sort(del)[3] mu - sort(del)[18] So
mu - sort(del)[3]
mu - sort(del)[18]
So we get the same answer they do. In particular I have the same 10th and 90th percentile. I want to point out that the range from the 10th to the 90th percentile is 3. This is the same as MIT has.
Hide
means
sort(means)
Hide means sort(means) I’mI’m getting different means. Important point- my 10th and 90th mean 38.9 and 41.9 . This is what I would expect. They are different because I am considering distances from 40.3, so I am reversing the subtraction order. Note that 40.3-38.9 = 1.4 (and 40.3 - 1.6 = 38.7). So what they call the percentile bootstrap gives a distribution that depends on the actual means we get and not the differences.
Key PointKey Point
The empirical bootstrap and the percentile bootstrap will be different in that what they call the empirical bootstrap will be the interval [x∗¯−δ.1,x∗¯−δ x∗¯−δ.9][x∗¯−δ.1,x∗¯−δ x∗¯−δ.9] whereas the percentile bootstrap will have the confidence interval [x∗¯−δ.9,x∗¯−δ x∗¯−δ.1][x∗¯−δ.9,x∗¯−δ x∗¯−δ.1]. Typically they shouldn’t be that different. I have my thoughts as to which I would prefer, but I am not the definitive source that OP requests.
Thought experiment- should the two converge if the sample size increases. Notice that there are 210210 possible samples of size 10. Let’s not go nuts, but what about if we take 2000 samples- a size usually considered sufficient.
Hide set.seed(1234) # reproducible boot.2k = matrix(NA,10,2000) for( i in c(1:2000)){ boot.2k[,i] = sample(orig.boot,10,replace = T) } mu2k = sort(apply(boot.2k,2,mean)) Let’s
set.seed(1234) # reproducible
boot.2k = matrix(NA,10,2000)
for( i in c(1:2000)){
boot.2k[,i] = sample(orig.boot,10,replace = TRUE)
}
mu2k = sort(apply(boot.2k, 2, mean))
Let’s look at mu2k
Hide
summary(mu2k)
mean(mu2k)-mu2k[200]
mean(mu2k) - mu2k[1801]
Hide summary(mu2k) mean(mu2k)-mu2k[200] mean(mu2k) - mu2k[1801] AndAnd the actual values-
Hide
mu2k[200]
mu2k[1801]
Hide mu2k[200] mu2k[1801] SoSo now what MIT calls the empirical bootstrap gives an 80% confidence interval of [,40.3 -1.87,40 40.3 +1.64] or [38.43,41 41.94] and the their bad percentile distribution gives [38.5,42] 42]. This of course makes sense because the law of large numbers will say in this case that the distribution should converge to a normal distribution. Incidentally, this is discussed in Efron and Hastie. The first method they give for calculating the bootstrap interval is to use mu =/- 1.96 sd. As they point out, for large enough sample size this will work. They then give an example for which n=2000$n=2000$ is not large enough to get an approximately normal distribution of the data.
