3
$\begingroup$

I just learned about the empirical bootstrap and how it can be used to estimate confidence intervals. However, does the bootstrap work for the degenerate case where the sample only contains one data point?

In other words, is the following a valid application of the empirical bootstrap? Let's assume that we want to use empirical bootstrap to find an 80% confidence interval for the mean:

  1. Sample a single data point $x_0$. The sample mean is $\bar{x} = x_0$.
  2. Resample for n rounds, resulting in n resamples $x_0^*, x_1^*, ..., x_n^*$, where each contains a single data point $x_0$.
  3. Compute the difference between each resample mean and the sample mean: $\delta_i^* = x_i^* - \bar{x} = x_0 - x_0 =0$.
  4. Rank the differences and take the 10th and 90th percentiles, $\delta_{0.9}^*$ and $\delta_{0.1}^*$, which are both 0.
  5. Compute the confidence interval as $[\bar{x} - \delta_{0.1}^*, \bar{x} - \delta_{0.9}^*] = [\bar{x}, \bar{x}] = [x_0, x_0]$.

The resulting confidence interval seems to implies a level of confidence that is incongruent with our intuition that a single data point would result in an unconfident estimation.

Is this a valid application of the bootstrap? If so, how do we know when to trust the results of the bootstrap - should we verify the sample size?

Reference: https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/readings/MIT18_05S14_Reading24.pdf.

$\endgroup$
3
$\begingroup$

The reference you included mentions this:

The bootstrap is based roughly on the law of large numbers, which says, in short, that with enough data the empirical distribution will be a good approximation of the true distribution. Visually it says that the histogram of the data should approximate the density of the true distribution.

This should give you the first hint that your example is probably not a good idea - the empirical distribution of a single data point is probably not a reasonable approximation to the "true" distribution, whatever that is.

Rubin's wonderful The Bayesian Bootstrap is a little more specific about some assumptions of the classical and Bayesian bootstraps:

...is it reasonable to use a model specification that effectively assumes all possible distinct values of X have been observed? Both the [Bayesian Bootstrap] and the [classical] Boostrap operate under this assumption. (p.4, Section 5; "Discussion of Model Specifications")

This should indicate in a straightforward way, I think, that your example is not a valid application of the bootstrap - unless the support of your random variable contains only $x_0$, your inference is making some curious model assumptions.

You may very reasonably point out that that's a surprising model specification - for example, it will never be satisfied for any continuous distribution! Rubin would, I think, agree with you, and point out that there's no quick fix here, along with the fact that blindly applying the bootstrap in all cases will hardly be a guarantee of good results. This assumption may be "close enough" for many cases, but a reasonable sample size and coverage over the support of the random variable are necessary to invoke a technique like this. He memorably concludes that article by noting that

Serious data analyses should always include serious consideration of model constraints...although the bootstrap and the BB may be useful in many particular contexts, there are no general data analytic panaceas that allow us to pull ourselves up by our bootstraps. (p. 5).

$\endgroup$
  • $\begingroup$ Thank you for the detailed answer! It makes sense that the bootstrap relies on the assumption that the empirical distribution is close to the true distribution, and that this assumption is violated when the sample is of size one. $\endgroup$ – deuxpierres Oct 13 at 6:11
  • $\begingroup$ You're quite welcome! $\endgroup$ – Louis Cialdella Oct 14 at 1:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.