Question
How do I perform a bootstrapped difference test when my bootstrapped sample sizes are smaller than the original sample sizes?
Typically I would do this by taking $p = \frac{\sum_{i=1}^{B} t_{i}^ {*} \geq t}{B}$, where
- $t_{i}^ {*}=\frac{\bar{x^ {*} }-\bar{y}^ {*}}{\sqrt{\sigma^{*2}_x/m + \sigma^{*2}_y/m}}$ for a pair of bootstrapped samples $x^{*}$, $y^{*}$
- $t=\frac{\bar{x}-\bar{y}}{\sqrt{\sigma^{2}_y/m + \sigma^{2}_x/m}}$
In this case, however, the obvious substitute for $t$ is
- $t=\frac{\bar{x}-\bar{y}}{\sqrt{\sigma^{2}_x/n_1 + \sigma^{2}_y/n_2}}$, where $m \neq n_1 \neq n_2$
I am unsure if this is valid. Any help on this would be much appreciated!
Background
I'm using bootstrapping to estimate if the respective population parameters for two populations $P_1$ and $P_2$ are significantly different.
For this purpose, I have samples from each population; namely, a sample $x_1$ of 900,000, and a sample $x_2$ of size 600,000. From these, I have done the following:
- I've computed test statistics $\bar{x_1}$ and $\bar{x_2}$¹, as well as their difference
- I've bootstrapped 2000 $x^∗_{1,i}$ 500,000-item samples and 2000 $x^{∗}_{2,i}$ 500,000-item samples
- I've computed 2000 test statistics $\overline{x^∗_{1,i}}$ and $\overline{x^∗_{2,i}}$, as well as the differences between each
¹These are not the means, but it is convenient to represent them this way. Anyone is free to change this if it is annoying.
