# Which randomization test is equivalent to bootstrapped CIs

I have used bootstrapping (percentile method) to calculate the confidence interval for the estimated mean of a set. I have now divided my data into two groups (a and b), and I want to test if the mean value in each group is different. I have considered the wilcoxon rank sum test, but would prefer a method similar to the bootstrap below.

library(boot)

set.seed(1)
value <- runif(100)
grp <- sample(c("a","b"), 100, replace = TRUE)

df <- data.frame(grp, value)

bootstrap_data <- function (data, func = mean, R = 10000) {
boot_func <- function(x, d) {
return(func(x[d]))
}

set.seed(1)
boot(data, boot_func, R = R)
}

bootstrap_ci <- function (data, func = mean, R = 10000) {
boot_data <- bootstrap_data(data, func, R)
boot.ci(boot_data, type="perc")
}

bootstrap_ci(value)
# 95%   ( 0.4648,  0.5700 )

• In what sense do you mean "equivalent"? Note that randomization tests are operating under the restriction that the null is true, while bootstrap intervals are not. Sep 18, 2016 at 6:13
• By equivalent i mean: A test that has the same relation to a bootstrap CI as a t-test has to a CI based on the t-distribution. Sep 18, 2016 at 14:48

You could bootstrap the difference of means:

library(boot)

set.seed(1)

grp <- sample(c("a","b"), 100, replace = TRUE)

#some data with an actual difference in means
value <- rnorm(100, mean = as.integer(factor(df$grp))) df <- data.frame(grp, value) b <- boot(df, function(DF, i) { DF <- DF[i,] mean(DF[DF$grp == "a", "value"]) -
mean(DF[DF$grp == "b", "value"]) }, R = 1e4, strata = as.integer(factor(df$grp)))

#bootstrap t-value
b$t0/sd(b$t)
#[1] -6.027335

#compare with t from normality assumption
t.test(value ~ grp, data = df)\$statistic
#        t
#-5.896552