Skip to main content
1 of 11
EdM
  • 101.5k
  • 11
  • 102
  • 303

If the null hypothesis doesn't hold, then bootstrap sampling is from a population described by the alternate hypothesis, not by the null hypothesis. Yet frequentist calculations of CIs and p-values are based on what would be expected if the null hypothesis holds.

In some circumstances, the percentile bootstrap can still serve for these purposes. As AdamO explains in this answer, there is a critical assumption needed about how the underlying distribution changes between populations described by the null and alternate hypotheses:

But one important assumption is that such a distribution is pivotal. This means that if the underlying parameter changes, the shape of the distribution is only shifted by a constant, and the scale does not necessarily change. This is a strong assumption!

The percentile bootstrap, happily, is only one of several ways to use bootstrapping to estimate CIs. The "empirical bootstrap" described in the class notes to which you linked is one example. Some are described in the answer I linked above and in other answers on that page; see this page for other perspectives and links to further information. The boot.ci() function in the R boot package provides 5 bootstrapping methods for CIs (including the percentile method).

You also can try other resampling techniques. For example, permutation testing can repeatedly shuffle class labels to represent populations described by the null hypothesis. This answer explains several types of resampling.

So the statement that "the percentile bootstrap should never be used" might be a bit of overstatement. Better to say that it can lead to problems and that it never needs to be used.

EdM
  • 101.5k
  • 11
  • 102
  • 303