# Question on the philosophy and functioning of hypothesis testing with parametric bootstrapping

In the book of Bradley Efron and Robert Tibshirani (1993) "An Introduction to the Bootstrap" chapter 16, a bootstrap method for hypothesis testing is presented. It is precised that the two quantities that have to be specified when carrying out a bootstrap hypothesis test are :

1. A test statistic $$t(X)$$,
2. A null distribution $$\hat{F}_0$$ for the data under the null hypothesis $$H_0$$.

$$X$$ represents the observed data and an estimate of the p-value is given by $$\hat{p} = \# \{t(X^b) \geq t(X) \}/B$$, where $$X^b$$ is the $$b$$-bootstrap resampling and $$B$$ is the number of bootstraps, i.e. $$b \in \{1,2,\cdots,B\}$$.

In some cases, $$t(X)$$ can be used as a parameter to define $$\hat{F}_0$$. Let us then write the 'bootstrap' distribution of $$X$$ under the null hypothesis $$\hat{F}_0(\cdot, t(X))$$. So each $$t(X^b)$$ is obtained from a sample $$X^b$$ drawn from $$\hat{F}_0(\cdot, t(X))$$.

Can someone explain why $$\hat{F}_0(\cdot, t(X))$$ is a good choice for the distribution under the null hypothesis? For me it looks like we would never be able to reject the null hypothesis because the probability to observe $$t(X^b)$$ close to $$t(X)$$ under $$\hat{F}_0(\cdot, t(X))$$ is really high. Is there something I am missing?

I don't find $$\hat F_0(\bullet,t(X))$$ suggested as null hypothesis in Chapter 16 of Efron and Tibshirani (1993). In general you need to know in advance what your null hypothesis of interest is, and then the bootstrap test null hypothesis needs to be chosen so that it is in line with this, as explained in the book. Indeed it doesn't make much sense to test the null hypothesis that a parameter is equal to $$t(X)$$ where $$t(X)$$ is the value of the statistic observed from your data. But then I don't think this is recommended anywhere.
What people in fact do, and this is mentioned in the book, is that they "translate* the empirical distribution so that it has the null hypothesis value of interest, i.e., if you want to test $$\mu=0$$ where $$\mu$$ is the mean of the underlying distribution, and say your $$t(X)=5$$ (observed mean on your data), then $$\hat F_0$$ would be the empirical distribution of your observations minus 5 (or if you run a parametric bootstrap, $$F_0(\bullet,0)$$, the distribution of your assumed parametric model at mean 0, potentially with estimated variance or other parameters).
• Yes, I agree this is mentionned in the book for the equality of means test. A bit further in the chapter, they test whether the distribution of a variable is unimodal or multimodal. To do so, they use gaussian kernel with $h$ the width parameter. They obtain $\hat{h}$ from the observations $X$ the minimum h such that the distribution is unimodal. Then, the null hypothesis distribution is chosen to be $\hat{F(\cdot,\hat{h}})$ and for $H_0: h \geq \hat{h}$. So, I understand that they did what I explained above. Am I missing something? Commented Jul 15, 2023 at 12:16
• @lulufofo OK, I see what you mean. The thing is that this doesn't work quite in the way you are proposing, but I admit it looks like that. The thing is that $\hat h$ is not a parameter that defines a parametric distribution here. $\hat h$ modifies the observed empirical distribution by smoothing it so much that it becomes unimodal, so what you call $F(\bullet,\hat h)$ is a unimodal distribution. But if the observed empirical distribution is in fact clearly not unimodal, observations drawn from $F(\bullet,\hat h)$ will look quite different from the data, and ... (to be continued) Commented Jul 15, 2023 at 13:39
• ...and if you "estimate" $\hat h^*$ from the bootstrapped data, it may look very different from $\hat h$ and the null hypothesis will be rejected. Note that $\hat h^*$ here does not estimate the same "parameter" as $\hat h$, because $F(\bullet,h)$ does not only depend on $h$ but also on the data set from which it was computed! This means that the new $F(\bullet,\hat h^*)$ may look very similar to the old $F(\bullet,\hat h)$ even if $\hat h$ and $\hat h^*$ are quite different (and the Fs may be different, if hs are similar). Commented Jul 15, 2023 at 13:42
• If the underlying true distribution is in fact $F(\bullet,\hat h)$ (unimodal!), we would expect $\hat h$ and $\hat h^*$ to be similar, and the bootstrap test wouldn't normally reject. So this is a special construction for this specific problem, having in mind the null hypothesis of interest, unimodality. There is no general principle that says that $F(\bullet,\hat h)$ is a good null hypothesis. Commented Jul 15, 2023 at 13:47