# Kullback–Leibler vs Kolmogorov-Smirnov distance

I can see that there are a lot of formal differences between Kullback–Leibler vs Kolmogorov-Smirnov distance measures. However, both are used to measure the distance between distributions.

• Is there a typical situation where one should be used instead of the other?
• What is the rationale to do so?

The KS-distance is typically used in the context of a non-parametric test. In fact, I've rarely seen it used as a generic "distance between distributions", where the $\ell_1$ distance, the Jensen-Shannon distance, and other distances are more common.
• Another use of KL-divergence worth mentioning is in hypothesis testing. Assume $X_1, X_2, \ldots$ are iid from measures with density either $p_0$ or $p_1$. Let $T_n = n^{-1} \sum_{i=1}^n \log( p_1(X_i) / p_0(X_i) )$. By Neyman--Pearson, an optimal test rejects when $T_n$ is large. Now, under $p_0$, $T_n \to -D(p_0 \,\vert\vert\, p_1)$ in probability and under $p_1$, $T_n \to D(p_1 \,\vert\vert\, p_0)$. Since $D(\cdot \,\vert\vert\, \cdot)$ is nonnegative, the implication is that using the rule $T_n > 0$ to reject $p_0$ is asymptotically perfect. – cardinal Apr 9 '11 at 2:06