# Glivenko-Cantelli Theorem

The Glivenko-Cantelli Theorem states that if $$F$$ is a distribution function, $$X_1,\dots,X_n \sim F$$, and $$\hat{F}_n$$ is the empirical distribution function, then $$\sup_{x \in \mathbb{R}} \lvert \hat{F}_n(x) - F(x) \rvert \xrightarrow{a.s.} 0 . \tag{1}$$

How does this differ from simply stating the following? $$\hat{F}_n(x) \xrightarrow{a.s.} F(x) \tag{2}$$

Using the definition of convergence almost surely, from (1): \begin{align*} \mathbb{P} \left( \lim_{n\rightarrow\infty} \left\lvert \sup_{x \in \mathbb{R}} \lvert \hat{F}_n(x) - F(x) \rvert \right\rvert = 0 \right) &= 1 \\ \mathbb{P} \left( \lim_{n\rightarrow\infty} \sup_{x \in \mathbb{R}} \lvert \hat{F}_n(x) - F(x) \rvert = 0 \right) &= 1 \tag{1a} \end{align*}

Using the definition of convergence almost surely, from (2): \begin{align*} \mathbb{P} \left( \lim_{n\rightarrow\infty} \left\lvert \hat{F}_n(x) - F(x) \right\rvert = 0 \right) = 1 \tag{2a} \end{align*}

To me, it seems that (1a) and (2a) are equivalent statements because of the least upper bound, and thus (1) and (2) are equivalent statements. But I have a feeling that I'm missing a subtle difference since otherwise I would think the Theorem would just be stated the simpler way (Equation (2)).

The differerence is in uniform convergence. 1 is saying that for all x there is a single n such that error is less than epsilon ( uniform). The other one is saying for each x there is a large enough n that error is less than epsilon(pointwise). An example from wikipedia of pointwise but not uniform convergence is $f_n (x)=x^n$ on $0\le x\le 1$. You need larger and larger n as you get closer to $x=1$.
• Hmm. Did you really mean for that to end with "closer to $x=0$"? I'd have thought that the need for larger $n$ was near $x=1$. Sep 22, 2014 at 22:04
It is perhaps worth noting that pointwise convergence of $$\hat F_n(x)$$ to $$F(x)$$ already implies uniform convergence where $$F$$ is continuous (because the cdfs are bounded and monotone). More precisely, if $$[a,b]$$ is an interval that does not contain any discontinuities of $$F$$, the convergence is uniform on $$[a,b]$$ -- and that's still true for $$a=-\infty$$ or $$b=\infty$$.
The conclusion of the Glivenko-Cantelli theorem is stronger: that the convergence is uniform even at discontinuities, and this is important. By contrast, if $$\hat F_n$$ are a sequence of empirical CDFs from distributions $$F_n$$ converging in distribution to $$F$$, we have pointwise convergence of $$\hat F_n(x)$$ to $$F(x)$$, and uniform convergence on intervals with no discontinuities, but not uniform convergence everywhere.