How to understand the uniform central limit theorem

Question

In a paper from Nickl I found a theorem (Theorem 4) with a form of central limit theorem $$\sqrt{n}(P_n-P)\rightarrow G$$ in $l^\infty(F)$ where $P$ is a law on $\mathbb{R}$, $F$ is a class of function and $G$ is a gaussian process indexed by $f\in F$. But the central limit theorem which I'm familiar with is of the form: $$\sqrt{n}(f_n-f)\rightarrow G$$ where $G$ in indexed by $x\in\mathbb{R}$. I am looking for help to understand the first version and also would like to know if one can transform it to the second version? Is it correct to demand $f\in F$ in the second version? Thanks a lot for any hint!

Answer 1

The first equation doesn't lead to the second (or vice versa) in any simple way. That's kind of the point of the paper: it observes that kernel density estimators work well for $L_1$ and integrated squared error summaries of pointwise error $f_n(x)-f(x)$ but wants to see how well they work measured in terms of getting large classes of expectations approximately correct. This is a difficult question precisely because there isn't a simple link.

Ok, so explaining the first equation. I will write $\mathbb{P}_n$ for the empirical measure because I like that notation ($P_n$ could be any sequence of measures)

Now, $Pf=E_{X\sim P}[f(X)]$, the mean of $f(X)$, and $\mathbb{P}_nf$ is the sample mean of $f(X)$. And $$\sqrt{n}(\mathbb{P}_nf-Pf)\stackrel{d}{\to} N(0,\sigma^2_f)$$ is then the classical central limit theorem for any single $f$. We can also write $\mathbb{G}_nf$ for the left-hand side and $Gf$ for the right-hand side, and it's just notation.

If ${\cal F}$ were a finite class we'd just have the classical multivariate CLT: stack the $f$s up to give a vector $F$ and say $$\sqrt{n}(\mathbb{P}_nF-PF)\stackrel{d}{\to} N(0,\Omega)$$ where $\Omega_{ij}=\mathrm{cov}[f_i(X),f_j(X)].$ We can still write $$\mathbb{G}_nF\stackrel{d}{\to}GF$$ where now we mean not only that the distributions are individually asymptotically Normal but that they arise as projections of a single multivariate Normal $G$. This doesn't add anything; finite collections that converge automatically do it uniformly.

When ${\cal F}$ is an infinite class, the theorem makes two claims. First, that the multivariate CLT holds for every finite subset of ${\cal F}$. $$\sqrt{n}(\mathbb{P}_nF-PF)\stackrel{d}{\to} N(0,\Omega)$$ Now when we write $$\mathbb{G}_n\stackrel{d}{\to}{\cal G}$$ we mean more: we mean that there's a tight Gaussian process ${\cal G}$ and all the finite-dimensional distributions $G$ arise as projections of it.

This does bring in the uniformity condition, which is not an issue in the finite case and which is what's really used to get the result in the paper.

The tightness condition in the uniform CLT implies (nearly implies?) stochastic equicontinuity $$\left|\mathbb{G}_nh_n-{\cal G}h\right|\stackrel{p}{\to}0$$ for any $h_n\stackrel{p}{\to} h$ all in ${\cal F}$.
Any class ${\cal F}$ for which this works is called a Donsker class.

That's all very well, and it tells you that expectations with respect to the density estimator agree well with the true expectations uniformly over ${\cal F}$, so you haven't broken the usual uniform approximation properties by smoothing. Why doesn't this tell you about $f_n(x)-f(x)$? Well, because that's not $(\mathbb{P}_n-P)g$ for any $g\in{\cal F}$. The problem isn't with $f_n$, which could well be an element of ${\cal F}$ (for example if ${\cal F}$ is functions of bounded variation). The problem is that evaluation at one $x$ isn't what the theorem is about.