UPDATE 06/2020: I just revisited this question and realised that there is a fairly clear cut answer. Specifically, the required condition is uniform integrability. Basically the class of functions of the distribution that are uniformly integrable can be in inferred using a bootstrap. Without uniform integrability, all bets are off. I'm fairly sure, although can't remember off the top of my head, that there is theorem in one of Brillinger's textbooks for exactly this situation.

Setup: Let $p_n(x) = \mathbb{P}(S_n \leq x)$ and let $p_n^*(x) = \mathbb{P}^*(S_n^* - S_n \leq x)$, where $S_n$ is some zero-mean test statistic that can validly be bootstrapped, eg a sample mean, and $S_n^*$ is a bootstrap re-sample of $S_n$. Note that $\mathbb{P}^*$ implies the probability is conditional on the underlying data. Assuming the conditions of some bootstrap theorem, eg stationary bootstrap, are fulfilled, we have the main bootstrap result:

\begin{equation} \sup_{x \in \mathbb{R}} | p_n(x) - p_n^*(x)| \overset{\mathbb{P}}{\rightarrow} 0, \: \textrm{as} \: n \rightarrow \infty \end{equation}

My question has two parts:

Question 1: The main bootstrap result is usually used to assert that bootstrapped confidence intervals are asymptotically valid. Can we make this assertion because of Slutsky's theorem? If so, why isn't continuity an issue?

Currently, my understanding is that the argument works as follows:

1) For some fixed point $x \in \mathbb{R}$, we use the above result to assert that $p_n^*(x) \overset{\mathbb{P}}{\rightarrow} p_n(x)$, as $n \rightarrow \infty$.

2)Use Slutsky's theorem to assert that $g(p_n^*(x)) \overset{\mathbb{P}}{\rightarrow} g(p_n(x))$, as $n \rightarrow \infty$, for some Borel function $g(a)$ that is continuous at $a$.

3) Choose $g(a)$ to be the quantile function, ie $g(F(x)) = \inf \{ x \in \mathbb{R} : \lambda \leq F(x) \}$ where $F(x)$ is a cdf and $\lambda \in (0, 1)$. The result then follows.

The problem with this argument as it stands is that, for general cdf functions, the quantile function is only guaranteed to be left continuous. I'm assuming that we get around this problem by exploiting the fact that $p_n(x)$ and $p_n^*(x)$ are both converging to the Normal, which has a continuous cdf, but I've never seen this stated formally anywhere, so I am wondering if my reasoning is actually correct.

Question 2: Having established that we can build asymptotically valid confidence intervals, I'm wondering what other characteristics of the distribution of $S_n$ we can consistently estimate?

I know that most bootstrap theorems also include a result of the form $\mathbb{V}^*S_n^* \overset{\mathbb{P}}{\rightarrow} \mathbb{V} S_n$, as $n \rightarrow \infty$, so that takes care of the variance. What about other moments? For example, can we infer from the main bootstrap result that $\mathbb{E}^* S_n^{*4} \overset{\mathbb{P}}{\rightarrow} \mathbb{E} S_n^4$, as $n \rightarrow \infty$? As in the case of confidence intervals, I tried to prove this using Slutsky's theorem, but was not comfortable with the resulting expression:

\begin{equation} \int x^4 dp_n^*(x) \overset{\mathbb{P}}{\rightarrow} \int x^4 dp_n(x), \: \mathrm{as} \: n \rightarrow \infty , \end{equation}

since it is not clear to me that we are not accumulating lots of "small" errors in the integral.

EDIT: I asked the question here since it is about the bootstrap. However, since it is also about probability theory, if users feel it would be more appropriate in stack exchange mathematics, please let me know.

  • $\begingroup$ I think almost any statstical functional can be validly boostrapped, with varying levels of confidence and precision. A key requirement is uniform convergence of the boostrap distribution for a statistic to the actual statistic. Also, jacknife-after-boostrap diagnostics can be done to determine how robust your statistic is. $\endgroup$ – user31668 Dec 5 '13 at 14:58
  • $\begingroup$ @Eupraxis1981 Thanks for the comment. Do you know of a formal proof, or reference to one? I've read a fair bit of the literature on this, eg Efron (1979), Politis and Romano (1994), Lahiri (2003), and no-one ever really offers a formal proof of convergence of characteristics of the distribution, other than the main result I mention in the question, which only covers convergence of the cdf at a given fixed point. Perhaps I'm being a bit dense, but I just don't see how it is obvious that this extends to convergence of other characteristics of the distribution, such as moments. $\endgroup$ – Colin T Bowers Dec 6 '13 at 0:34
  • 1
    $\begingroup$ Assuming that the actual sampling distribution converges uniformly in n to the true sampling distribution, then any function of this distributoion will also converge, since you can bring the limit under the integral in the functional via uniform convergene theorem. $\endgroup$ – user31668 Dec 6 '13 at 1:28
  • $\begingroup$ @Eupraxis1981 Thanks, that definitely got me reading in the right direction. One minor point still worries me. My reading of the uniform convergence theorem suggests that it is used when the function in question is the integrand. But in my example above, $p_n^*(x)$ is the integrator. Does the theorem still apply in this situation? I can see that it should if $p_n^*(x)$ is everywhere differentiable, but that condition may not hold for general cdf's. If you can respond to this and turn it into an answer, I'll tick and upvote. Thanks. $\endgroup$ – Colin T Bowers Dec 6 '13 at 4:11
  • $\begingroup$ I've posted an answer based on our exchange. I think the mention of the uniform convergence theorem took this too far afield. There are much simpler reasons why you can calculate almost any other statistic of the sampling distribution using bootstrapping. $\endgroup$ – user31668 Dec 6 '13 at 16:36

The essense of the nonparametric boostrap method is to use the EDF as if it were the true distribution and then perform monte carlo sampling/analysis to the EDF. Therefore, as long as the EDF is a good representation of the true CDF, the boostrap sampling distribution will be a good approximation of the true sampling distribution, and hence any statistics dervied from this approximate sampling distribution will be apprximately correct to the degree that your original data accurately depict the true underlying distribution. Hence, as the sample size gets bigger, both the EDF and the associated sample statistics converge to the true values. The convergence therorem I cited is useful for continuous functions, but its really not needed. All we need to know is that as the sample size gets bigger, the statistical error in treating the EDF as the CDF approaches zero by the law of large numbers. Therefore, simulation from this estimated distribution converges to simulation from the true distribution as the sample size grows.

The major caveat to this is when your sampling statsistic does not uniformly converge to the true value, with maximial order statsitics being a classic example.

Therefore, the answer to your question is that any quantity of your sampling distribution can be estimated from the boostrap sampling distribution as long as the bootstrap sampling distribution itself converges uniformly to the true sampling distribution. Below are some links, some of which contain further links to very interesting papers, on when this condition is not met.

This paper will also be interesting for you. the encyclopedia of mathematics has a good entry on boostrap failure as well. This has also been discussed previously on Cross validated: What are examples where a "naive bootstrap" fails?

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.