Is there a result that provides the bootstrap is valid if and only if the statistic is smooth?

Question

Throughout we assume our statistic $\theta(\cdot)$ is a function of some data $X_1, \ldots X_n$ which is drawn from the distribution function $F$; the empirical distribution function of our sample is $\hat{F}$. So $\theta(F)$ is the statistic viewed as a random variable and $\theta(\hat{F})$ is the bootstrap version of the statistic. We use $d_\infty$ as the KS distance

There are "if and only if" results for the validity of the bootstrap if the statistic is a simple linear statistic. For example Theorem 1 from Mammen "When does the bootstrap work?"

If $\theta(F) = \frac{1}{n} \sum_{i-1}^n h_n(X_i)$ for some arbitrary function $h_n$ then the bootstrap works in the sense that $$d_\infty\big[\mathscr{L}(\theta(\hat{F})-\hat{t}_n), \mathscr{L}(\theta(F)-t_n)\big] \underset{p}{\rightarrow} 0$$ if and only if there exists $\sigma_n$ and $t_n$ such that $$d_\infty\big[\mathscr{L}(\theta(F)-t_n), N(0, \sigma_n^2)\big]\underset{p}{\rightarrow} 0$$ Where we can define $\hat{t_n}$ as some function of our sample and $t_n = \mathbb{E}(\hat{t}_n)$

There are also more general results that the bootstrap works for general statistics, for example Theorem 1.6.3 from Subsampling by Politis Romano and Wolf:

Assume $F$ is drawn from the class of all distributions with finite support. Assume the statistic $\theta(\cdot)$ is Frechet differentiable at $F$ with respect to the supremum norm and the derivative $g_F$ satisfies $0 < \textrm{Var}_F[g_F(x)] < \infty$. Then $\theta(F)$ is asymptotically normal and the bootstrap works in the sense of the previous theorem.

I would like an `if and only if' version of the second theorem. This will require a notion of smoothness different from Frechet differentiability because Politis, Romano and Wolf (1999) show that the sample median is not Frechet differentiable but the bootstrap still works. However the sample median is still a smooth function of the data.

There are some informal comments in Mammen that smoothness is necessary:

Typically local asymptotic linearity seems to be necessary for consistency of bootstrap

The citation is to:

van Zwet, W (1989). Talk given at the conference on "Asymptotic methods for computer intensive procedures in statistics" in Olberwolfach.

But I can find no trace of this talk apart from a handful of citations.

Answer 1

$\blacksquare$ (1)Why quantile estimators are not Frechet differentiable but their bootstrap estimator is still consistent?

You need Hadamard differentialbility(or compact differentiability depending on your reference source) as a sufficient condition to make bootstrap work in that case, the median and any quantile is Hadamard differentiable. Frechet differentiability is too strong in most applications.

Since usually it suffices to discuss a Polish space, there you want a locally linear functional to apply a typical compactness argument to extend your consistency result to the global situation. Also see the lineariazation comment below.

Theorem 2.27 of [Wasserman] will give you an intuition how Hadamard derivative is a weaker notion. And Theorem 3.6 and 3.7 of [Shao&Tu] will give sufficient condition for weak consistency in terms of $\rho$-Hadamard differentiability of the statistical functional $T_{n}$ with observation size $n$.

$\blacksquare$ (2)What will affect the consistency of bootstrap estimators?

[Shao&Tu]pp.85-86 illustrated situations where inconsistency of bootstrap estimators may occur.

(1)The bootstrap is sensitive to the tail behavior of the population $F$. The consistency of $H_{BOOT}$ requires moment conditions that are more stringent than those needed for the existence of the limit of $H_0$.

(2)The consistency of bootstrap estimator requires a certain degrees of smoothness from the given statistic (functional) $T_{n}$.

(3)The behavior of the bootstrap estimator sometimes depends on the method used to obtain bootstrap data.

And in Sec 3.5.2 of [Shao&Tu] they revisited the quantile example using a smoothing kernel $K$. Notice that moments are linear functionals, the quote in your question "Typically local asymptotic linearity seems to be necessary for consistency of bootstrap" is requiring some level of analyticity of the functional, which might be necessary because if that fails you can create some pathological case like Weierstrass function(which is continuous yet nowhere differentiable).

$\blacksquare$ (3)Why local linearity seems necessary in ensuring the consistency of bootstrap estimator?

As for the comment "Typically local asymptotic linearity seems to be necessary for consistency of bootstrap" made by Mammen as you mentioned. A comment from [Shao&Tu]p.78 is as following, as they commented the (global) linearization is only a techinique that facilitates the proof of consistency and does not indicate any necessity:

Linearization is another important technique in proving the consistency of bootstrap estimators, since results for linear statistics are often available or may be established using the techniques previously introduced. Suppose that a given statistic Tn can be approximated by a linear random variable $\bar{Z_n}=\frac{1}{n}\sum_{i=1}^{n}\phi(X_n)$ (where $\phi(X)$ is a linear statistic in $X$), i.e., (3.19)$$T_n=\theta+\bar{Z_n}+o_{P}(\frac{1}{\sqrt{n}})$$ Let $T_n^{*}$ and $\bar{Z_n^{*}}$ be the bootstrap analogs of $T_n$ and $\bar{Z_n}$, respectively, based on the bootstrap sample $\{X_1^{*},\cdots,X_n^{*}\}$. If we can establish a result for $T_n^{*}$ similar to (3.19), i.e., (3.20)$$T_n^{*}=\theta+\bar{Z_n}^{*}+o_{P}(\frac{1}{\sqrt{n}})$$ then the limit of $H_{BOOT}(x)$(where $x$ is the value of parameter)$=P\{\sqrt{n}(T_n-T_n^{*}) \leq x\}$ is the same as that of $P\{\sqrt{n}(\bar{Z_n}-\bar{Z_n}^{*}) \leq x\}$.We have thus reduced the problem to a problem involving a "sample mean" $\bar{Z_n}$, whose bootstrap distribution estimator can be shown to be consistent using the methods in Sections 3.1.2-3.1.4.

And they gave an example 3.3 of obtaining the bootstrap consistency for MLE type bootstrapping. However if global linearity is effective in that way, it is hard to imagine how one would prove consistency without local linearity. So I guess that is what Mammen wanted to say.

$\blacksquare$ (4)Further comments

Beyond the discussion provided by [Shao&Tu] above, I think what you want is a characterization condition of consistency of bootstrap estimators.

Pitifully, I do not know one characterization of consistency of a bootstrap estimator for a very general class of distribution in $M(X)$. Even if there is one I feel it requires not only smoothness of $T$. But there does exist characterization for a certain class of statistical models like $CLT$ class in [Gine&Zinn]; or commonly compactly supported class(directly from above discussion) defined over a Polish space.

Plus, the Kolmogorov-Smirnov distance, according to my taste is the wrong distance if our focus is classic asymptotics(in contrast to "uniform" asymptotics for empirical processes). Because KS-distance does not induce the weak topology which is a natural ground for study of asymptotic behavior, the weak topology on the space $M(X)$ is induced by bounded Lipschitz distance(OR Prohorov-Levy distance) as adopted by [Huber] and many other authors when the focus is not empirical process. Sometimes the discussion of limiting behavior of empirical process also involve BL-distance like[Gine&Zinn].

I hate to be cynical yet I still feel that this is not the only statistical writing that is "citing from void". By saying this I simply feel the citation to van Zwet's talk is very irresponsible although van Zwet is a great scholar.

$\blacksquare$ Reference

[Wasserman]Wasserman, Larry. All of Nonparametric Statistics, Springer, 2010.

[Shao&Tu]Shao, Jun, and Dongsheng Tu. The jackknife and bootstrap. Springer, 1995.

[Gine&Zinn]Giné, Evarist, and Joel Zinn. "Bootstrapping general empirical measures." The Annals of Probability (1990): 851-869.

[Huber]Huber, Peter J. Robust statistics. Wiley, 1985.