Kullback–Leibler vs Kolmogorov-Smirnov distance I can see that there are a lot of formal differences between Kullback–Leibler vs Kolmogorov-Smirnov distance measures.
However, both are used to measure the distance between distributions.


*

*Is there a typical situation where one should be used instead of the other? 

*What is the rationale to do so?

 A: Another way of stating the same thing as the previous answer in more layman terms:
KL Divergence - Actually provides a measure of how big of a difference are two distributions from each other. As mentioned by the previous answer, this measure isnt an appropriate distance metric since its not symmetrical. I.e. distance between distribution A and B is a different value from the distance between distribution B and A.
Kolmogorov-Smirnov Test - This is an evaluation metric that looks at greatest separation between the cumulative distribution of a test distribution relative to a reference distribution. In addition, you can use this metric just like a z-score against the Kolmogorov distribution to perform a hypothesis test as to whether the test distribution is the same distribution as the reference.  This metric can be used as a distance function as it is symmetric. I.e. greatest separation between CDF of A vs CDF of B is the same as greatest separation between CDF of B vs CDF of A.
A: The KL-divergence is typically used in information-theoretic settings, or even Bayesian settings, to measure the information change between distributions before and after applying some inference, for example. It's not a distance in the typical (metric) sense, because of lack of symmetry and triangle inequality, and so it's used in places where the directionality is meaningful. 
The KS-distance is typically used in the context of a non-parametric test. In fact, I've rarely seen it used as a generic "distance between distributions", where the $\ell_1$ distance, the Jensen-Shannon distance, and other distances are more common. 
A: KL divergence upper bounds Kolmogrov Distance and Total variation, meaning that if two distributions $\mathcal{D}_1, \mathcal{D}_2$ have a small KL divergence, then it follows that $\mathcal{D}_1, \mathcal{D}_2$ have a small total variation and subsequently a small Kolmogrov distance (in that order).
Also check out this paper for more information - On Choosing and Bounding Probability Metrics, by Gibbs and Su.
A: KS test and KL divergence test both are used to find the difference between two distributions
KS test is statistical-based and KL divergence is information theory-based
But the one major diff between KL and KS test, and why KL is more popular in machine learning is because the formulation for KL divergence is differentiable. And for solving optimization problems in machine learning we need a function to be differentiable.
In the context of machine learning, KL_dist(P||Q) is often called the information gain achieved if Q is used instead of P
links:
https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test
https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test
