# When KL Divergence and KS test will show inconsistent results?

I know that Kullback–Leibler divergence and Kolmogorov–Smirnov test are different and should be used in different scenarios. But they are similar in many ways and given two distributions, we could calculate their KL divergence in terms of bits and p-value under KS test (and there are also other metrics like Jensen–Shannon divergence and many other hypothesis testing methods. But Let's just talk about KL divergence and KS test here.)

My question is: Under what circumstances, that KS test will provide a very small p-value but KL divergence will give us a very small distance? What is the intuition behind it? It would be better if there could be any concrete examples.

• For a comparison of Kolmogorov distance and KL divergence note that Kolmogorov distance compares values of cumulative distribution functions while KL divergence compares (log) ratio of densities. – kjetil b halvorsen Jun 27 '18 at 8:27

[If you don't wish to confound the distinction between Kolmogorov-Smirnov distance and p-value with the difference in what the two things are looking at, it might be better to explore the differences in the two measures ($D_{KS}$ and $D_{KL}$) directly, but that's not what is being asked here.]