Suppose I have a small number of samples drawn from an unknown distribution $\{X_1,X_2,...,X_n\}$, where $0\le X_i \le L$, and $3\le n \le10$. I want to identify a metric to understand how far these samples are from a uniform distribution over the length $L$.

One of the metrics commonly used for this purpose is the Kullback-Leibler divergence which finds the "distance" of a given probability distribution from another. In this case, I want to find how different is the sample of size $n$ from a uniform distribution using a single number.

  • $\begingroup$ The Kullback-Leibrer divergence is a single number. Why do you want something else? $\endgroup$ – Alecos Papadopoulos Oct 29 '20 at 21:21
  • $\begingroup$ KL divergence works best when there are a large number of samples available. I was trying to find a metric which would work for smaller number of samples. $\endgroup$ – smog Oct 29 '20 at 22:43
  • $\begingroup$ Since you know the target distribution exactly, what about a Kolmogorov-Smirnov test for goodness-of-fit? Then the single number you are after is a p-value. $\endgroup$ – soakley Nov 2 '20 at 23:44
  • $\begingroup$ @soakley, thanks for the comment! The K-S test seems like the best option to go ahead with. Although I am planning on using it in a slightly modified manner. $\endgroup$ – smog Nov 3 '20 at 18:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.