Why does the supremum of the Brownian bridge have the Kolmogorov–Smirnov distribution?

Question

The Kolmogorov–Smirnov distribution is known from the Kolmogorov–Smirnov test. However, it is also the distribution of the supremum of the Brownian bridge.

Since this is far from obvious (to me), I would like to ask you for an intuitive explanation of this coincidence. References are also welcome.

Answer 1

$\sqrt{n}\sup_x|F_n-F|=\sup_x|\frac{1}{\sqrt{n}}\sum_{i=1}^nZ_i(x)| $

where $Z_i(x)=1_{X_i\leq x}-E[1_{X_i\leq x}]$

by CLT you have $G_n=\frac{1}{\sqrt{n}}\sum_{i=1}^nZ_i(x)\rightarrow \mathcal{N}(0,F(x)(1-F(x)))$

this is the intuition...

brownian bridge $B(t)$ has variance $t(1-t)$ http://en.wikipedia.org/wiki/Brownian_bridge replace $t$ by $F(x)$. This is for one $x$...

You also need to check the covariance and hence it still is easy to show (CLT) that for ($x_1,\dots,x_k$) $(G_n(x_1),\dots,G_n(x_k))\rightarrow (B_1,\dots,B_k)$ where $(B_1,\dots,B_k)$ is $\mathcal{N}(0,\Sigma)$ with $\Sigma=(\sigma_{ij})$, $\sigma_{ij}=\min(F(x_i),F(x_j))-F(x_i)F(x_j)$.

The difficult part is to show that the distribution of the suppremum of the limit is the supremum of the distribution of the limit... Understanding why this happens requires some empirical process theory, reading books such as van der Waart and Welner (not easy). The name of the Theorem is Donsker Theorem http://en.wikipedia.org/wiki/Donsker%27s_theorem ...

Answer 2

For Kolmogorov-Smirnov, consider the null hypothesis. It says that a sample is drawn from a particular distribution. So if you construct the empirical distribution function for $n$ samples $f(x) = \frac{1}{n} \sum_i \chi_{(-\infty, X_i]}(x)$, in the limit of infinite data, it will converge to the underlying distribution.

For finite information, it will be off. If one of the measurements is $q$, then at $x=q$ the empirical distribution function takes a step up. We can look at it as a random walk which is constrained to begin and end on the true distribution function. Once you know that, you go ransack the literature for the huge amount of information known about random walks to find out what the largest expected deviation of such a walk is.

You can do the same trick with any $p$-norm of the difference between the empirical and underlying distribution functions. For $p=2$, it's called the Cramer-von Mises test. I don't know the set of all such tests for arbitrary real, positive $p$ form a complete class of any kind, but it might be an interesting thing to look at.