0
$\begingroup$

I have two independent i.i.d., respectively $Y_1,\dots,Y_{n_Y} \sim F_Y$ of size $n_Y$ and $X_1,\dots,X_{n_X} \sim G_X$ of size $n_X$. I would like to test: $$ H_0:\mu_X = \mu_Y $$ versus any alternative hypothesis like: $$ H_1:\mu_X \gtreqless \mu_Y $$ with bootstrap. My question is if the following two alternatives for calculating the bootstrap p-values are the same:

  • bootstrap $Y$ generating independent bootstrap i.i.d. samples $Y^*$ of size $n_Y$ then bootstrap $X$ generating independent bootstrap i.i.d. samples $X^*$ of size $n_X$ and calculate the test statistic;
  • concatenate $X$ and $Y$ ($XY = \begin{bmatrix} X\\ Y \end{bmatrix}$) and bootstrap $XY$ generating independent bootstrap i.i.d. samples $XY^*$ of size $n_Y+n_X$ and calculate the test statistic;

I don't understand why and if the second option makes sense.

In this code from the book Bootstrap Methods With Applications in R by Gerhard Dikta and Marsel Scheer (2021) I think they consider the second option:

twoSampleLocModBootpValue <-function(x, y, alternative = c("two-sided", "less", "greater"), R = 999){

# x -observed data (first sample)
# y -observed data (second sample)
# alternative -specifies the alternative hypothesis
# R -number of MC simulations

alternative <-match.arg(alternative)

n1 <-length(x)
n2 <-length(y)

# test statistic
tstat <-function(d, i){
boot.xy <-d[i]

x <-boot.xy[1:n1]
y <-boot.xy[-(1:n1)]
s <-sqrt( ((n1-1) * var(x) + (n2 -1) * var(y)) / (n1 + n2 -2))
mu.x <-mean(x)
mu.y <-mean(y)
(mu.x -mu.y) / s / sqrt(1 / n1 + 1 / n2)
}

xy <-c(x,y)

# test statistics for the observed data
t0 <-tstat(xy, 1:(n1 + n2))

#Rresampledteststatistics
bt<-boot::boot(xy,tstat,R=R)$t[,1]

#return p-value
if(alternative=="greater") return(c(pvalue=mean(bt>t0)))
if(alternative=="less") return(c(pvalue=mean(bt<t0)))
c(pvalue=mean(abs(bt)>abs(t0)))
}

Then applying the function to datasets such that the null hypothesis is true:

#H0c orrect, mu_x equals mu_y
set.seed(123,kind="Mersenne-Twister",normal.kind="Inversion") 
twoSampleLocModBootpValue(rnorm(10,mean=3,sd=2), rnorm(20,mean=3,sd=2), alternative="greater")

##pvalue
##0.3143143
$\endgroup$

2 Answers 2

2
$\begingroup$

Either of those approaches could be appropriate, depending on the initial sampling scheme. Standard bootstrapping from a data sample should represent the same type of sampling as was done to obtain the original sample from the full population.

So, if you had two separate populations $X$ and $Y$ and deliberately chose respectively $n_X$ and $n_Y$ separately from each of those populations, then the first approach is appropriate.

Instead, say that you sampled $n$ members of a single population who were distinguished only thereafter by their characteristics $X$ and $Y$. In that case, you ended up with $n_X$ and $n_Y$ based on the random sampling. Then the second approach best represents the way you collected the data sample.

$\endgroup$
1
  • 1
    $\begingroup$ For example, if you obtained experimental data where X was the outcome of the control group and Y of the treatment - in such a case, you would rather want to use method 1 - else outcomes from treated individuals might sometimes also be sampled 'into' the control group and vice versa. $\endgroup$
    – A.Fischer
    Dec 21, 2021 at 16:25
0
$\begingroup$

Method 1 determines the sampling distribution of the test statistic under the observed data distributions. I'm not sure this is particularly useful. I suppose it can give you a 95% confidence interval for the test statistic. I would use Method 1 to construct a 95% confidence interval for the difference in means rather than perform a hypothesis test.

Often for a hypothesis test you want to determine the sampling distribution of the test statistic under then NULL hypothesis. Then comparing the observed test statistic to the bootstrapped null distribution gives you a p-value.

Method 2 potentially gives you the desired sampling distribution. Randomly sample with replacement nX observations from the combined set (XY) to be your bootstrap resample from X and sample with replacement nY observations from the combined set (XY) to be your bootstrap resample from Y. Compute your test statistic. Note, muX = muY for these resamples. Repeat 1000 times and compare your observed test statistic to the 1000 simulated test statistics.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.