I'm looking for references (textbook if possible) that treat the strong consistency of bootstrapped maximum likelihood (MLH) or more generally M-estimators. By strong consistency I mean that the difference between the cdf of the original statistic and the bootstrap counterpart goes a.s. to zero uniformly (see below).

So far I have been following the textbook:

Shao, Tu: The Jacknife and Bootstrap. 1995.

that provides a really nice account of the bootstrap and theorems of this type. For example, see Theorem 3.1 (page 80) in Shao & Tu, denote by $\bar{X}_n$ the sample mean of $X_1,\ldots,X_n$ (i.i.d.), and by $h\colon \mathbb{R}\to\mathbb{R}$ a function such that $h'(\mu) \neq 0$ and $h'$ is continuous at $\mu$, where $\mu = \mathbb{E}[X]$ and $\mathbb{E}[X^2] < \infty$. For a statistic $$ T_n = h(\bar{X}_n) $$ with bootstrap counterpart $T^*_n$ we then have $$\Vert F_{T_n^*} - F_{T_n} \Vert_{\infty} = \sup_{x\in\mathbb{R}} \vert F_{T_n^*}(x) - F_{T_n}(x) \vert \xrightarrow[n\to\infty]{a.s.} 0. $$ Here $F_{T_n^*}$ and $F_{T_n}$ are the respective distribution functions and the convergence a.s. is necessary since $F_{T_n^*}$ is random.

This settles method of moment type estimators (in fact in Shao & Tu the general multivariate version is considered), but I'm wondering about implicitly defined estimators like MLH (or M-estimators).

My question is therefore, if $\mathcal{M} = \{f_{\theta};\theta\in\Theta\}$ is a parametric statistical model and $\widehat{\theta}_n$ is the MLH estimator based on $X_1,\ldots,X_n$ (i.i.d. with density $f_{\theta_0}$) what can be said about the limit $$\Vert F_{\widehat{\theta}_n^*} - F_{\widehat{\theta}_n} \Vert_{\infty}.$$

I'm looking for references that either give conditions and a proof, or, preferably, a textbook that outlines the results in a digestible way.


It depends on what you count as 'digestible', but I like the version in van der Vaart's Asymptotic Statistics, which develops consistency of the bootstrap from the delta method in both the finite-dimensional and infinite-dimensional case.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.