How to bootstrap confidence interval for large dataset

Suppose we want to calculate confidence interval of mean value. If the dataset is massive ($n$ samples), classical bootstrapping is hard to apply since the size of the resample must be $n$.

But if I resample the dataset with a smaller size $k$, the variance of the bootstrap distribution will be different from the original one. Can I simply divide it by $\frac{n}{k}$? Is there some advice to choose $k$?

Thanks.

• See Hastie and Efron's new book Computer Age Statistical Inference where these issues are dealt with in depth. Jan 15, 2017 at 13:59
• There is a bootstrap method called m out of n bootstrap where the bootstrap sample size is m<n for n being the sample size of the original data set. Whether or not this helps you with very large n I am not sure because it requires that m/n does not go to zero with large n. so you can't conveniently take a small m and a very large n. Jan 15, 2017 at 15:46
• Why do you want to estimate the variance? You state you actually want the confidence interval. Jan 15, 2017 at 16:28
• @mdewey It's actually used to evaluate an AB test, so I use a t-test after estimating the variance of each experiment group. Jan 16, 2017 at 2:39

Although the classical bootstrap requires you to resample for $n$ samples, but this is always not necessary especially if you have a large data set.
You should set $k$ to as large as you can handle computationally.
If your k ($n$ in the formula) is large enough, your sample mean should be unbiased and your confidence interval should be small enough practically.
• Thanks for reply. One more question: can I estimate the standard error of the original $n$ samples as $\frac{s \sqrt{k}}{\sqrt{n}}$, where $s$ is the SE of bootstrap distribution. Jan 16, 2017 at 2:42