15
$\begingroup$

When searching the internet on the sample size of for boostrap samples I generally find that the size should equal the original sample size. However, I can't find any explanation on why this should be the case.

  • In what way is it bad to use a smaller sample size?
  • Is it always better to use a larger sample size?

Some of the sites which report this: http://www.stata.com/support/faqs/statistics/bootstrapped-samples-guidelines/ http://www2.stat.duke.edu/courses/Fall12/sta101.002/Sec3-34.pdf

$\endgroup$
4
  • 2
    $\begingroup$ Those sites also contain the answer to your question. Can you clarify why you are not satisfied with those answers? $\endgroup$ Feb 23 '17 at 22:49
  • $\begingroup$ @MaartenBuis The links only say we should do it with the original sample size, but didn't explain or give a proof. $\endgroup$
    – SmallChess
    Feb 23 '17 at 22:59
  • 10
    $\begingroup$ At heart you're trying to infer something about the behaviour of some aspect of the sampling distribution of some statistic of interest (e.g. a standard error) at the sample size you have. $\endgroup$
    – Glen_b
    Feb 24 '17 at 1:05
  • 3
    $\begingroup$ Quoted from your first link: " However, the standard error estimates are dependent upon the number of observations in each replication. Here, on average, we would expect the variance estimate of _b[foreign] to be twice as large for a sample of 37 observations than that for 74 observations. This is due mainly to the form of the variance of the sample mean, s2/n." $\endgroup$ Feb 24 '17 at 8:26
16
$\begingroup$

This is from Efron and Tibshirani's An Introduction to the Bootstrap (first sentence of Chapter 2):

The bootstrap is a computer-based method for assigning measures of accuracy to statistical estimates.

This suggests that we should in some way respect the correct sample size $n$: The accuracy of statistical estimates depends on the sample size, and your statistical estimate will come from a sample of size $n$.


How to estimate the standard error of the mean with the bootstrap, and how you're fooling yourself if you draw bootstrap samples of any other size than $n$.

We understand the behavior of the mean very well. As you'll no doubt remember from intro to stats, the standard error of the mean depends on the sample size, $n$, in the following manner: $SEM = s/\sqrt{n}$, where $s^2$ is the sample variance.

The bootstrap principle is that a bootstrap sample relates to your sample as your sample relates to the population. In other words you're assuming your sample is a pretty good approximation to the population and that you can use it as a proxy. Let $x^{*b}$ denote the $b$-th bootstrap sample, and let $\hat \mu^*_b$ be the mean of this bootstrap sample. The bootstrap estimate of standard error is:

$$ SE_\mathrm{boot} = \sqrt{\frac{\sum_{b=1}^B (\hat \mu^*_b - \bar \mu^*)^2}{(B-1)}} $$ where $B$ is the number of bootstrap samples you've drawn (the more the merrier,) and $\bar \mu^* = \sum \hat \mu^*_b/B$ is the average of the bootstrapped means. This is a long way of saying that the bootstrap estimate of standard error is simply the sample standard deviation of the bootstrapped statistics. You're using the spread in the bootstrapped means to say something about the accuracy of the sample mean.

Now, we're bootstrapping, so we're treating the original sample as a population: it is a discrete distribution with mass $1/n$ at each data point $x_i$. We can draw as many samples from this as we want, and in principle we can make them as large or small as we want. If we draw an $n^*$-sized bootstrap sample and estimate its mean $\hat \mu^*$, we know that $\hat \mu^* \sim N(\hat \mu, s/\sqrt{n^*})$. For $n^*=n$ the standard deviation of your bootstrapped mean is exactly the central limit theorem-dictated $SEM$ for the original sample. This isn't true for any other $n^*$.

So in this example, if $n^* = n$, the sample standard deviation of $\{\hat \mu^*_b\}$ is a good representation of the correct standard error of the mean. If you draw larger bootstrap samples you get really good estimates of the sample mean, but their spread no longer directly relates to the standard error you're trying to estimate because you can make their distribution arbitrarily tight.

$\endgroup$
2
  • 2
    $\begingroup$ For other problems where the bootstrap doesn't work well modifications like m out of n work. $\endgroup$ Apr 25 '17 at 13:45
  • 1
    $\begingroup$ Note that this is basically the same answer as in the stats.stackexchange.com/questions/268024/… which I was not aware of when writing this $\endgroup$
    – einar
    Apr 26 '17 at 14:16

Not the answer you're looking for? Browse other questions tagged or ask your own question.