# What characteristics of the distribution of a test statistic can be inferred using a bootstrap?

UPDATE 06/2020: I just revisited this question and realised that there is a fairly clear cut answer. Specifically, the required condition is uniform integrability. Basically the class of functions of the distribution that are uniformly integrable can be in inferred using a bootstrap. Without uniform integrability, all bets are off. I'm fairly sure, although can't remember off the top of my head, that there is theorem in one of Brillinger's textbooks for exactly this situation.

Setup: Let $$p_n(x) = \mathbb{P}(S_n \leq x)$$ and let $$p_n^*(x) = \mathbb{P}^*(S_n^* - S_n \leq x)$$, where $$S_n$$ is some zero-mean test statistic that can validly be bootstrapped, eg a sample mean, and $$S_n^*$$ is a bootstrap re-sample of $$S_n$$. Note that $$\mathbb{P}^*$$ implies the probability is conditional on the underlying data. Assuming the conditions of some bootstrap theorem, eg stationary bootstrap, are fulfilled, we have the main bootstrap result:

$$$$\sup_{x \in \mathbb{R}} | p_n(x) - p_n^*(x)| \overset{\mathbb{P}}{\rightarrow} 0, \: \textrm{as} \: n \rightarrow \infty$$$$

My question has two parts:

Question 1: The main bootstrap result is usually used to assert that bootstrapped confidence intervals are asymptotically valid. Can we make this assertion because of Slutsky's theorem? If so, why isn't continuity an issue?

Currently, my understanding is that the argument works as follows:

1) For some fixed point $$x \in \mathbb{R}$$, we use the above result to assert that $$p_n^*(x) \overset{\mathbb{P}}{\rightarrow} p_n(x)$$, as $$n \rightarrow \infty$$.

2)Use Slutsky's theorem to assert that $$g(p_n^*(x)) \overset{\mathbb{P}}{\rightarrow} g(p_n(x))$$, as $$n \rightarrow \infty$$, for some Borel function $$g(a)$$ that is continuous at $$a$$.

3) Choose $$g(a)$$ to be the quantile function, ie $$g(F(x)) = \inf \{ x \in \mathbb{R} : \lambda \leq F(x) \}$$ where $$F(x)$$ is a cdf and $$\lambda \in (0, 1)$$. The result then follows.

The problem with this argument as it stands is that, for general cdf functions, the quantile function is only guaranteed to be left continuous. I'm assuming that we get around this problem by exploiting the fact that $$p_n(x)$$ and $$p_n^*(x)$$ are both converging to the Normal, which has a continuous cdf, but I've never seen this stated formally anywhere, so I am wondering if my reasoning is actually correct.

Question 2: Having established that we can build asymptotically valid confidence intervals, I'm wondering what other characteristics of the distribution of $$S_n$$ we can consistently estimate?

I know that most bootstrap theorems also include a result of the form $$\mathbb{V}^*S_n^* \overset{\mathbb{P}}{\rightarrow} \mathbb{V} S_n$$, as $$n \rightarrow \infty$$, so that takes care of the variance. What about other moments? For example, can we infer from the main bootstrap result that $$\mathbb{E}^* S_n^{*4} \overset{\mathbb{P}}{\rightarrow} \mathbb{E} S_n^4$$, as $$n \rightarrow \infty$$? As in the case of confidence intervals, I tried to prove this using Slutsky's theorem, but was not comfortable with the resulting expression:

$$$$\int x^4 dp_n^*(x) \overset{\mathbb{P}}{\rightarrow} \int x^4 dp_n(x), \: \mathrm{as} \: n \rightarrow \infty ,$$$$

since it is not clear to me that we are not accumulating lots of "small" errors in the integral.

EDIT: I asked the question here since it is about the bootstrap. However, since it is also about probability theory, if users feel it would be more appropriate in stack exchange mathematics, please let me know.

• I think almost any statstical functional can be validly boostrapped, with varying levels of confidence and precision. A key requirement is uniform convergence of the boostrap distribution for a statistic to the actual statistic. Also, jacknife-after-boostrap diagnostics can be done to determine how robust your statistic is.
– user31668
Dec 5, 2013 at 14:58
• @Eupraxis1981 Thanks for the comment. Do you know of a formal proof, or reference to one? I've read a fair bit of the literature on this, eg Efron (1979), Politis and Romano (1994), Lahiri (2003), and no-one ever really offers a formal proof of convergence of characteristics of the distribution, other than the main result I mention in the question, which only covers convergence of the cdf at a given fixed point. Perhaps I'm being a bit dense, but I just don't see how it is obvious that this extends to convergence of other characteristics of the distribution, such as moments. Dec 6, 2013 at 0:34
• Assuming that the actual sampling distribution converges uniformly in n to the true sampling distribution, then any function of this distributoion will also converge, since you can bring the limit under the integral in the functional via uniform convergene theorem.
– user31668
Dec 6, 2013 at 1:28
• @Eupraxis1981 Thanks, that definitely got me reading in the right direction. One minor point still worries me. My reading of the uniform convergence theorem suggests that it is used when the function in question is the integrand. But in my example above, $p_n^*(x)$ is the integrator. Does the theorem still apply in this situation? I can see that it should if $p_n^*(x)$ is everywhere differentiable, but that condition may not hold for general cdf's. If you can respond to this and turn it into an answer, I'll tick and upvote. Thanks. Dec 6, 2013 at 4:11
• I've posted an answer based on our exchange. I think the mention of the uniform convergence theorem took this too far afield. There are much simpler reasons why you can calculate almost any other statistic of the sampling distribution using bootstrapping.
– user31668
Dec 6, 2013 at 16:36