Bootstrap for small sample

Question

Does bootstrap method help for small sample? In my mind, bootstrap is a solution when you don't have belief in a normality assumption. If your data is random enough, it might be convincing to sample from your sample and get an empirical distribution for your statistic.

But if the size of your sample is small, even if you believe your sample is random, can you say bootstrap is the best way for small sample? Would there be a chance that your bootstrap result is biased?

Answer 1

The validity of the bootstrap is based on asymptotic arguments, so there is not much basis to say that bootstrap is the best way for a small sample.

However, there are some situations where there is good reason to prefer bootstrap to usual asymptotic inference for small (or really any) samples. Some uses of bootstrap achieve what is called an asymptotic refinement. This refers to a situation where the bootstrap estimate of the distribution of your estimator converges faster than the usual asymptotic approximation. For example, if you have some asymptotically normal estimate, $\sqrt{n}(\hat{\theta} - \theta) \leadsto_d N(0,\sigma^2)$, then $$P( \sqrt{n}(\hat{\theta}-\theta)/\hat{\sigma} < z) = \Phi(z) + O(n^{-1}),$$ i.e. the size distortion of hypothesis tests decreases at rate $1/n$. Some versions of bootstrap can be shown to have size distortions that decrease at a rate faster than $1/n$. Specifically, using bootstrap to compute the distribution of a pivotal statistic (one whose asymptotic distribution is completely known, like the t-statistic) generally gives this faster convergence. In this example if for each bootstrap replicate of the data, you calculate $t_b = \sqrt{n}(\hat{\theta}_b - \hat{\theta})/\hat{\sigma}_b$, and let $\hat{F}$ be the CDF of the $t_b$, then
$$P( \sqrt{n}(\hat{\theta}-\theta)/\hat{\sigma} < z) = \hat{F}(z) + o(n^{-1})$$ Here we have a little $o$ where before there was a big $O$. This gives some hope that when bootstrapping pivotal statistics, the bootstrap might be more accurate for small samples than the usual asymptotic approximations.

Answer 2

Taking this to extreme, imagine you have only two samples. Say that you take 1000 bootstrap samples by sampling with replacement samples of size two from your data. Could this give you any more detailed representation of distribution of the statistic of interest, then the two sales alone? Obviously not. Bootstrap needs relatively large samples, the more complicated is your data, the larger samples you need.