Glivenko-Cantelli Theorem

Question

The Glivenko-Cantelli Theorem states that if $F$ is a distribution function, $X_1,\dots,X_n \sim F$, and $\hat{F}_n$ is the empirical distribution function, then $$\sup_{x \in \mathbb{R}} \lvert \hat{F}_n(x) - F(x) \rvert \xrightarrow{a.s.} 0 . \tag{1}$$

How does this differ from simply stating the following? $$\hat{F}_n(x) \xrightarrow{a.s.} F(x) \tag{2}$$

Using the definition of convergence almost surely, from (1): $$\begin{align*} \mathbb{P} \left( \lim_{n\rightarrow\infty} \left\lvert \sup_{x \in \mathbb{R}} \lvert \hat{F}_n(x) - F(x) \rvert \right\rvert = 0 \right) &= 1 \\ \mathbb{P} \left( \lim_{n\rightarrow\infty} \sup_{x \in \mathbb{R}} \lvert \hat{F}_n(x) - F(x) \rvert = 0 \right) &= 1 \tag{1a} \end{align*}$$

Using the definition of convergence almost surely, from (2): $$\begin{align*} \mathbb{P} \left( \lim_{n\rightarrow\infty} \left\lvert \hat{F}_n(x) - F(x) \right\rvert = 0 \right) = 1 \tag{2a} \end{align*}$$

To me, it seems that (1a) and (2a) are equivalent statements because of the least upper bound, and thus (1) and (2) are equivalent statements. But I have a feeling that I'm missing a subtle difference since otherwise I would think the Theorem would just be stated the simpler way (Equation (2)).

Answer 1

The differerence is in uniform convergence. 1 is saying that for all x there is a single n such that error is less than epsilon ( uniform). The other one is saying for each x there is a large enough n that error is less than epsilon(pointwise). An example from wikipedia of pointwise but not uniform convergence is $ f_n (x)=x^n $ on $0\le x\le 1$. You need larger and larger n as you get closer to $x=1$.

Answer 2

It is perhaps worth noting that pointwise convergence of $\hat F_n(x)$ to $F(x)$ already implies uniform convergence where $F$ is continuous (because the cdfs are bounded and monotone). More precisely, if $[a,b]$ is an interval that does not contain any discontinuities of $F$, the convergence is uniform on $[a,b]$ -- and that's still true for $a=-\infty$ or $b=\infty$.

The conclusion of the Glivenko-Cantelli theorem is stronger: that the convergence is uniform even at discontinuities, and this is important. By contrast, if $\hat F_n$ are a sequence of empirical CDFs from distributions $F_n$ converging in distribution to $F$, we have pointwise convergence of $\hat F_n(x)$ to $F(x)$, and uniform convergence on intervals with no discontinuities, but not uniform convergence everywhere.