# Convergence Issues for Bootstrap Distributions

the following is part of a proof from van der Vaarts book on asymptotic statistics:

I want to show that if for a continuous distribution function F

$$P\left(\frac{\hat{\theta}-\theta}{\hat{\sigma}}\leq x\mid P\right) \rightarrow F(x) \mbox{ and } P\left(\frac{\hat{\theta}^\ast-\theta^\ast}{\hat{\sigma}^\ast}\leq x\mid \hat{P}\right)\rightarrow F(x) \text{ (in probability)}$$ $\quad\forall x$

then

$$\sup_x\| P\left(\frac{\hat{\theta}-\theta}{\hat{\sigma}}\leq x\mid P\right)-P\left(\frac{\hat{\theta}^\ast-\theta^\ast}{\hat{\sigma}^\ast}\leq x\mid \hat{P}\right) \| \rightarrow 0 \text{ (in probability)}$$

So i think it should go like this:

Define $X=\frac{\hat{\theta}_n-\theta}{\hat{\sigma}_n}$ and $\hat{X}=\frac{\hat{\theta}^\ast_n-\hat{\theta}_n}{\hat{\sigma}_n}$. \begin{align*} &\sup_x \| P\left(X\leq x\mid P\right)-P\left(\hat{X}\leq x\mid \hat{P}\right) \| \\ \leq&\sup_x\left(\| P\left(X\leq x\mid P\right)-F\|+ \|F-P\left(\hat{X}\leq x\mid \hat{P}\right) \| \right)\\ \leq&\sup_x \| P\left(X\leq x\mid P\right)-F \| +\sup_x \|P\left(\hat{X}\leq x\mid \hat{P}\right)-F\| \end{align*}

and now i'm a bit lost... i think the claim should follow because the sup is attained because F is continous and monotonous or something like that. i feel it is close but

Hint: mimic the proof that a tight family of pdf's that converges pointwise to a continuous limit converges uniformly. (note by OP: german speaking readers may also find a proof of $F_n \rightarrow F$ weakly \Rightarrow \sup \|F_n - F | \rightarrow 0) in Bauer - Mass- und Integrationstheorie)

In my mind, Helly's selection theorem and your problem were related.

Here is a solution. Let $\epsilon \gt 0$. Since $F$ is continuous with limits at infinity, we can find a partition of $\mathbb{\bar{R}}$ of the form: $a_{0} = -\infty \lt a_{1}\lt\dots \lt a_{k}\lt a_{k+1}=+\infty$ such that
\begin{align*} & F(a_{i}) \lt F(a_{i-1}) + \epsilon \end{align*} for $i=1,\dots,k+1$.

We have: \begin{align} &\sup_x|\hat{F}(x) - \hat{F}^*(x)| = \max_i \sup_{x \in [a_{i-1},a_{i})}|\hat{F}(x) - \hat{F}^*(x)| , \end{align}

but for $x \in [a_{i-1},a_i)$ we have

\begin{align} &-|\hat{F}^*(a_{i-1}) - \hat{F}(a_i)| \le \\ &\le \hat{F}^*(a_{i-1}) - \hat{F}(a_i) \le\hat{F}^*(x) - \hat{F}(x)\le\hat{F}^*(a_i) - \hat{F}(a_{i-1}) \le \\ & \le |\hat{F}^*(a_i) - \hat{F}(a_{i-1})| \end{align}

so

$|\hat{F}^*(x) - \hat{F}(x)|\le max\{|\hat{F}^*(a_i) - \hat{F}(a_{i-1})|,|\hat{F}^*(a_{i-1}) - \hat{F}(a_i)|\} \\ \stackrel{(a)}{\le} max\{|\hat{F}^*(a_{i-1}) - F(a_{i-1})|, |\hat{F}(a_{i-1} - F(a_{i-1})|, |\hat{F}^*(a_{i}) - F(a_{i})|, |\hat{F}(a_{i}) - F(a_{i})|, |F(a_i) - F(a_{i-1})|\}$.

By the selection of $\{a_i\}_i$, $|F(a_i) - F(a_{i-1})| \le \epsilon$.

By assumption, for each $i=1,\dots , k$ we can find a value of the of the index that parametrizes the convergence in probability (which is implicit in your statement, and that I will refer to "the sample size") such that beyond that value:
\begin{align*} & |\hat{F}(a_i) - F(a_i) | \le \epsilon \end{align*} for $i=1, \dots ,k$. Therefore, if the sample size is large enough:

\begin{align} &\sup_x|\hat{F}(x) - \hat{F}^*(x)| \stackrel{(b)}{\le} \max_i \{|\hat{F}^*(a_{i}) - F(a_{i})|\} + 2\epsilon . \end{align}

Let $\delta \gt 0$. By the assumption of convergence in probability (as the sample size increases), if the sample size sufficiently large: \begin{align*} & P( |\hat{F}^*(a_i) - F(a_i) | \ge \epsilon ) \le \frac{\delta}{k} \end{align*} for $i=1, \dots ,k$.

Finally, \begin{align*} & P( \sup_x|\hat{F}(x) - \hat{F}^*(x)| \ge 3\epsilon ) \le \\ & P( \max_i \{|\hat{F}^*(a_{i}) - F(a_{i})|\} + 2\epsilon \ge 3\epsilon ) = \\ & P( \max_i \{|\hat{F}^*(a_{i}) - F(a_{i})|\} \ge \epsilon ) \le \\ & \sum_{i=1}^k P(|\hat{F}^*(a_{i}) - F(a_{i})| \ge \epsilon) \le \sum_{i=1}^k \frac{\delta}{k} = \delta. \end{align*}

Since $\delta \gt 0$ is arbitrary, this shows:
\begin{align*} & \lim P( \sup_x|\hat{F}(x) - \hat{F}^*(x)| \ge 3\epsilon ) = 0 , \end{align*} and since $\epsilon \gt 0$ is arbitrary, this show: \begin{align*} & \sup_x|\hat{F}(x) - \hat{F}^*(x)| \leadsto 0 . \end{align*}

• Very nice, thank you. I worked through it and can see what you are doing, but i have some questions: - With "value of the of the index that parametrizes the convergence in probability" you mean sth like $F_n$? - the inequality sign marked (a) is a bit of a mystery to me. i toy around with triangle inequality a bit, but i dont get to the other side - With the one marked (b) it is clear where the 2 $\epsilon$ come from. The other part is somehow because of the monotonicity i guess? other than than i like the proof. – BootstrapBill Aug 18 '14 at 22:10
• You are right, (a) should be corrected. "<= 3*" instead of "<=" seem to work, but then one has to change some constants. In your statement you are taking limit over some index, which is not written explicitly; I mean that index; since your questions is in the context of bootstrap, it is probably the size of the original sample. – VictorZurkowski Aug 21 '14 at 19:10