# Sample size is $10^7$, what happens if we bootstrap with replacement using subsample of size $5000$?

Let $$\{X_1,X_2,\dots,X_n\}$$ be a sample of $$n$$ iid observations of a random variable $$X$$, and let $$\overline X_n = \frac{1}{n} \sum_{i=1}^n X_i$$ be the sample mean.

Now suppose we want to use bootstrapping to estimate, for example, the variance of the sample mean. Let $$\{X_1^{*(i)},X_2^{*(i)},\dots,X_n^{*(i)}\}$$ be a sample with replacement from the original sample and let $$\overline X_n^{*(i)} = \frac{1}{n} \sum_{j=1}^n X_j^{*(i)}$$ be the sample mean for the $$i$$-th bootstrap sample.

Let $$B$$ be the number of bootstrap rounds. Then we have a bootstrap estimate of the variance. $$\text{Var}_{B,n} = \frac{1}{B-1} \sum_{i=1}^B (\overline X_n^{*(i)} - \overline X_B^*)^2, \quad \quad \text{where} \ \overline X_B^* = \frac{1}{B} \sum_{i=1}^B \overline X_n^{*(i)}.$$

Suppose $$n$$ is very large and computing the bootstrap variance estimate will be computationally intensive. For example suppose we were using a slow computer and take $$n=10^7$$

To speed things up, we decide that instead of repeatedly drawing samples of size $$n$$ we will instead draw samples of size $$m \ll n$$ with replacement. For example, for $$n=10^7$$, we could take $$m=5000$$.

Let $$\{Y_1^{*(i)},Y_2^{*(i)},\dots,Y_m^{*(i)}\}$$ be a sample with replacement from the original sample $$\{X_1,X_2,\dots,X_n\}$$ and let $$\overline Y_m^{*(i)} = \frac{1}{m} \sum_{j=1}^m Y_j^{*(i)}$$ be the sample mean for the $$i$$-th bootstrap sample.

Now, this time we have the variance estimate: $$\text{Var}_{B,m} = \frac{1}{B-1} \sum_{i=1}^B (\overline Y_m^{*(i)} - \overline Y_B^*)^2, \quad \quad \text{where} \ \overline Y_B^* = \frac{1}{B} \sum_{i=1}^B \overline Y_m^{*(i)}.$$

Is $$\text{Var}_{B,m}$$ a valid estimate of the variance of the sample mean? One issue that I have noticed is that since $$m$$ is such so much smaller than $$n$$ that we have effectively sampled without replacement when we computed $$\text{Var}_{B,m}$$. Is this a problem?

• I must be missing something as$$\text{var}({\bar X}_n)=\frac{1}{n}\text{var}(X_i)$$and$$\text{var}{(\bar Y}_m)\approx\frac{1}{m}\text{var}(X_i)$$ – Xi'an Apr 5 at 14:06
• "Now suppose we want to use bootstrapping to estimate, for example, the variance of the sample mean" Which variance do you wish to estimate? The variance of the sample $X_1,\dots,X_n$ or the variance of the population from which the sample $X_1,\dots,X_n$ is obtained? It makes no sense to use bootstrapping to estimate $\bar{X}$ and $\text{var}(X)$ when you can just as well use the sample $X_1,\dots,X_n$. The bootstrapping does not add any information. – Sextus Empiricus Apr 5 at 14:59
• @SextusEmpiricus I am talking about estimating the variance of the sample mean, in analogy to estimating the variance of the sample median on the second page of these notes – ManUtdBloke Apr 5 at 15:52
• @SextusEmpiricus I realise the bootstrap doesn't add any information, my post is a simple toy example to frame my question about what happens if we perform bootstrapping where we repeatedly draw samples of size $m$ with replacement from an original dataset of size $n$, in contrast to the standard bootstrap which repeatedly draws samples of size $n$ with replacement from the original dataset of size $n$. – ManUtdBloke Apr 5 at 15:54