the following is part of a proof from van der Vaarts book on asymptotic statistics:

I want to show that if for a continuous distribution function F

$$P\left(\frac{\hat{\theta}-\theta}{\hat{\sigma}}\leq x\mid P\right) \rightarrow F(x) \mbox{ and } P\left(\frac{\hat{\theta}^\ast-\theta^\ast}{\hat{\sigma}^\ast}\leq x\mid \hat{P}\right)\rightarrow F(x) \text{ (in probability)}$$ $\quad\forall x$


$$\sup_x\| P\left(\frac{\hat{\theta}-\theta}{\hat{\sigma}}\leq x\mid P\right)-P\left(\frac{\hat{\theta}^\ast-\theta^\ast}{\hat{\sigma}^\ast}\leq x\mid \hat{P}\right) \| \rightarrow 0 \text{ (in probability)}$$

So i think it should go like this:

Define $X=\frac{\hat{\theta}_n-\theta}{\hat{\sigma}_n}$ and $\hat{X}=\frac{\hat{\theta}^\ast_n-\hat{\theta}_n}{\hat{\sigma}_n}$. \begin{align*} &\sup_x \| P\left(X\leq x\mid P\right)-P\left(\hat{X}\leq x\mid \hat{P}\right) \| \\ \leq&\sup_x\left(\| P\left(X\leq x\mid P\right)-F\|+ \|F-P\left(\hat{X}\leq x\mid \hat{P}\right) \| \right)\\ \leq&\sup_x \| P\left(X\leq x\mid P\right)-F \| +\sup_x \|P\left(\hat{X}\leq x\mid \hat{P}\right)-F\| \end{align*}

and now i'm a bit lost... i think the claim should follow because the sup is attained because F is continous and monotonous or something like that. i feel it is close but


1 Answer 1


Hint: mimic the proof that a tight family of pdf's that converges pointwise to a continuous limit converges uniformly. (note by OP: german speaking readers may also find a proof of $F_n \rightarrow F$ weakly \Rightarrow \sup \|F_n - F | \rightarrow 0) in Bauer - Mass- und Integrationstheorie)

In my mind, Helly's selection theorem and your problem were related.

Here is a solution. Let $\epsilon \gt 0$. Since $F$ is continuous with limits at infinity, we can find a partition of $\mathbb{\bar{R}}$ of the form: $a_{0} = -\infty \lt a_{1}\lt\dots \lt a_{k}\lt a_{k+1}=+\infty$ such that
\begin{align*} & F(a_{i}) \lt F(a_{i-1}) + \epsilon \end{align*} for $i=1,\dots,k+1$.

We have: \begin{align} &\sup_x|\hat{F}(x) - \hat{F}^*(x)| = \max_i \sup_{x \in [a_{i-1},a_{i})}|\hat{F}(x) - \hat{F}^*(x)| , \end{align}

but for $x \in [a_{i-1},a_i)$ we have

\begin{align} &-|\hat{F}^*(a_{i-1}) - \hat{F}(a_i)| \le \\ &\le \hat{F}^*(a_{i-1}) - \hat{F}(a_i) \le\hat{F}^*(x) - \hat{F}(x)\le\hat{F}^*(a_i) - \hat{F}(a_{i-1}) \le \\ & \le |\hat{F}^*(a_i) - \hat{F}(a_{i-1})| \end{align}


$|\hat{F}^*(x) - \hat{F}(x)|\le max\{|\hat{F}^*(a_i) - \hat{F}(a_{i-1})|,|\hat{F}^*(a_{i-1}) - \hat{F}(a_i)|\} \\ \stackrel{(a)}{\le} max\{|\hat{F}^*(a_{i-1}) - F(a_{i-1})|, |\hat{F}(a_{i-1} - F(a_{i-1})|, |\hat{F}^*(a_{i}) - F(a_{i})|, |\hat{F}(a_{i}) - F(a_{i})|, |F(a_i) - F(a_{i-1})|\} $.

By the selection of $\{a_i\}_i$, $|F(a_i) - F(a_{i-1})| \le \epsilon $.

By assumption, for each $i=1,\dots , k$ we can find a value of the of the index that parametrizes the convergence in probability (which is implicit in your statement, and that I will refer to "the sample size") such that beyond that value:
\begin{align*} & |\hat{F}(a_i) - F(a_i) | \le \epsilon \end{align*} for $i=1, \dots ,k$. Therefore, if the sample size is large enough:

\begin{align} &\sup_x|\hat{F}(x) - \hat{F}^*(x)| \stackrel{(b)}{\le} \max_i \{|\hat{F}^*(a_{i}) - F(a_{i})|\} + 2\epsilon . \end{align}

Let $\delta \gt 0$. By the assumption of convergence in probability (as the sample size increases), if the sample size sufficiently large: \begin{align*} & P( |\hat{F}^*(a_i) - F(a_i) | \ge \epsilon ) \le \frac{\delta}{k} \end{align*} for $i=1, \dots ,k$.

Finally, \begin{align*} & P( \sup_x|\hat{F}(x) - \hat{F}^*(x)| \ge 3\epsilon ) \le \\ & P( \max_i \{|\hat{F}^*(a_{i}) - F(a_{i})|\} + 2\epsilon \ge 3\epsilon ) = \\ & P( \max_i \{|\hat{F}^*(a_{i}) - F(a_{i})|\} \ge \epsilon ) \le \\ & \sum_{i=1}^k P(|\hat{F}^*(a_{i}) - F(a_{i})| \ge \epsilon) \le \sum_{i=1}^k \frac{\delta}{k} = \delta. \end{align*}

Since $\delta \gt 0$ is arbitrary, this shows:
\begin{align*} & \lim P( \sup_x|\hat{F}(x) - \hat{F}^*(x)| \ge 3\epsilon ) = 0 , \end{align*} and since $\epsilon \gt 0$ is arbitrary, this show: \begin{align*} & \sup_x|\hat{F}(x) - \hat{F}^*(x)| \leadsto 0 . \end{align*}

  • $\begingroup$ Very nice, thank you. I worked through it and can see what you are doing, but i have some questions: - With "value of the of the index that parametrizes the convergence in probability" you mean sth like $F_n$? - the inequality sign marked (a) is a bit of a mystery to me. i toy around with triangle inequality a bit, but i dont get to the other side - With the one marked (b) it is clear where the 2 $\epsilon$ come from. The other part is somehow because of the monotonicity i guess? other than than i like the proof. $\endgroup$ Commented Aug 18, 2014 at 22:10
  • $\begingroup$ You are right, (a) should be corrected. "<= 3*" instead of "<=" seem to work, but then one has to change some constants. In your statement you are taking limit over some index, which is not written explicitly; I mean that index; since your questions is in the context of bootstrap, it is probably the size of the original sample. $\endgroup$ Commented Aug 21, 2014 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.