The Glivenko-Cantelli Theorem states that if $F$ is a distribution function, $X_1,\dots,X_n \sim F$, and $\hat{F}_n$ is the empirical distribution function, then $$\sup_{x \in \mathbb{R}} \lvert \hat{F}_n(x) - F(x) \rvert \xrightarrow{a.s.} 0 . \tag{1}$$

How does this differ from simply stating the following? $$\hat{F}_n(x) \xrightarrow{a.s.} F(x) \tag{2}$$

Using the definition of convergence almost surely, from (1): $$\begin{align*} \mathbb{P} \left( \lim_{n\rightarrow\infty} \left\lvert \sup_{x \in \mathbb{R}} \lvert \hat{F}_n(x) - F(x) \rvert \right\rvert = 0 \right) &= 1 \\ \mathbb{P} \left( \lim_{n\rightarrow\infty} \sup_{x \in \mathbb{R}} \lvert \hat{F}_n(x) - F(x) \rvert = 0 \right) &= 1 \tag{1a} \end{align*}$$

Using the definition of convergence almost surely, from (2): $$\begin{align*} \mathbb{P} \left( \lim_{n\rightarrow\infty} \left\lvert \hat{F}_n(x) - F(x) \right\rvert = 0 \right) = 1 \tag{2a} \end{align*}$$

To me, it seems that (1a) and (2a) are equivalent statements because of the least upper bound, and thus (1) and (2) are equivalent statements. But I have a feeling that I'm missing a subtle difference since otherwise I would think the Theorem would just be stated the simpler way (Equation (2)).


2 Answers 2


The differerence is in uniform convergence. 1 is saying that for all x there is a single n such that error is less than epsilon ( uniform). The other one is saying for each x there is a large enough n that error is less than epsilon(pointwise). An example from wikipedia of pointwise but not uniform convergence is $ f_n (x)=x^n $ on $0\le x\le 1$. You need larger and larger n as you get closer to $x=1$.

  • 3
    $\begingroup$ Hmm. Did you really mean for that to end with "closer to $x=0$"? I'd have thought that the need for larger $n$ was near $x=1$. $\endgroup$
    – Glen_b
    Commented Sep 22, 2014 at 22:04

It is perhaps worth noting that pointwise convergence of $\hat F_n(x)$ to $F(x)$ already implies uniform convergence where $F$ is continuous (because the cdfs are bounded and monotone). More precisely, if $[a,b]$ is an interval that does not contain any discontinuities of $F$, the convergence is uniform on $[a,b]$ -- and that's still true for $a=-\infty$ or $b=\infty$.

The conclusion of the Glivenko-Cantelli theorem is stronger: that the convergence is uniform even at discontinuities, and this is important. By contrast, if $\hat F_n$ are a sequence of empirical CDFs from distributions $F_n$ converging in distribution to $F$, we have pointwise convergence of $\hat F_n(x)$ to $F(x)$, and uniform convergence on intervals with no discontinuities, but not uniform convergence everywhere.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.