I am working on using a bootstrap technique to compute confidence intervals of a parameter of interest.

Let $\textbf{Z}_1, ... \textbf{Z}_n\in\mathbb{R}^d$ be an (iid) random sample. From this random sample, we compute $\hat{\theta}$ parameter (very ugly to express explicitly). I would like to obtain CI for this parameter.

I utilize the basic bootstrap technique. This involves randomly sampling, with replacement, from our random sample to generate multiple bootstrap samples, each matching the size of our original dataset. For each bootstrap sample, I calculate the estimate of the statistic $\hat{\theta}^\star$. Subsequently, we determine the $95%$-quantiles of the re-sampled statistics to derive the confidence intervals. I want to prove their asymptotic correctness. Is there some general result that I can use? I only found theoretical justification using Berry-Essen theorem for $\theta$=mean, but my statistic $\theta$ is much more complicated.

In summary, I want to prove something like this:

Theorem Let $\hat{\theta}$ be an estimator (from a sample size $n$) of $\theta$ and let $\mathbb{E}||\textbf{Z}||^2<\infty$.

Denote the resamples from the original sample as $(\textbf{Z}^\star_{1,1}, \dots \textbf{Z}^\star_{1,n}),\dots, (\textbf{Z}^\star_{B,1}, \dots, \textbf{Z}^\star_{B,n})$, with corresponding estimates $\hat{\theta}_1^\star, \dots, \hat{\theta}_B^\star$ for $B\in\mathbb{N}$.

Let $U:=\hat{\theta}_{(\alpha)}^\star$ represent the $B(1-\alpha)$ largest value out of $\hat{\theta}_1^\star, \dots, \hat{\theta}_B^\star$.

$$\lim_{n\to\infty}\lim_{B\to\infty}P(\hat{\theta}<U) =\alpha. $$


1 Answer 1


How you prove this depends on your statistic $\theta$; there are complicated and straightforward versions depending on how difficult $\theta$ is. Fundamentally, though, the bootstrap works by the delta method; there's no need for the Berry-Esseen theorem.

A simplifying fact is that $\hat\theta$ is going to have a Normal distribution, so getting the tail probabilities asymptotically correct just implies getting the asymptotic mean and variance parameters correct

Often, your statistic $\hat\theta$ will be a differentiable function of a mean. In that case, you prove that the bootstrap is correct for the mean, and it transfers to being correct for the statistic automatically. Or your $\hat\theta$ solves $$\frac{1}{n}\sum_{i=1}^n U_i(\theta)=0$$ and $U_i$ has some regularity conditions that make $\hat\theta$ asymptotically Normal. Again, you show the bootstrap is correct for the mean, and the same arguments that make $\hat\theta$ asymptotically Normal also show the bootstrap is correct.

There's a more general argument that requires more background maths. If your data are from an iid sequence in $\mathbb{R}^d,$ then the empirical CDF is asymptotically Normal: $$\sqrt{n}(\mathbb{F}_n-F)\stackrel{w}{\to} Z$$ where $Z$ is a Gaussian process indexed by $\mathbb{R}^d.$ The delta method then says that any suitably differentiable function $\theta(\mathbb{F}_n)$ is asymptotically Normal. Since the bootstrap is correct for $\mathbb{F}_n$, in the sense that $$\sqrt{n}(\mathbb{F}^*_n-\mathbb{F})\stackrel{w}{\to} Z$$ for the same limiting $Z$ (for almost all data sequences), the delta method also says the bootstrap is correct.

A good source for this with the details is Chapter 23 of Asymptotic Statistics by van der Vaart. He starts off with the mean, and then the delta-method in finite-dimensional cases, and then the infinite-dimensional case. Finally, he talks about higher-order accuracy for sufficiently well-behaved statistics through the Edgeworth expansion.

  • $\begingroup$ Thanks! I read the chapter of van der Vaart, indeed it is the best possible reference to my question. If you are interested, I provide more information about my problem here: stats.stackexchange.com/q/639016/348308 $\endgroup$ Feb 10 at 19:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.