I'm trying to come up with a metric for measuring non-uniformity of a distribution for an experiment I'm running. I have a random variable that should be uniformly distributed in most cases, and I'd like to be able to identify (and possibly measure the degree of) examples of data sets where the variable is not uniformly distributed within some margin.

An example of three data series each with 10 measurements representing frequency of the occurrence of something I'm measuring might be something like this:

a: [10% 11% 10%  9%  9% 11% 10% 10% 12%  8%]
b: [10% 10% 10%  8% 10% 10%  9%  9% 12%  8%]
c: [ 3%  2% 60%  2%  3%  7%  6%  5%  5%  7%]   <-- non-uniform
d: [98% 97% 99% 98% 98% 96% 99% 96% 99% 98%]

I'd like to be able to distinguish distributions like c from those like a and b, and measure c's deviation from a uniform distribution. Equivalently, if there's a metric for how uniform a distribution is (std. deviation close to zero?), I can perhaps use that to distinguish ones with high variance. However, my data may just have one or two outliers, like the c example above, and am not sure if that will be easily detectable that way.

I can hack something to do this in software, but am looking for statistical methods/approaches to justify this formally. I took a class years ago, but stats is not my area. This seems like something that should have a well-known approach. Sorry if any of this is completely bone-headed. Thanks in advance!


4 Answers 4


If you have not only the frequencies but the actual counts, you can use a $\chi^2$ goodness-of-fit test for each data series. In particular, you wish to use the test for a discrete uniform distribution. This gives you a good test, which allows you to find out which data series are likely not to have been generated by a uniform distribution, but does not provide a measure of uniformity.

There are other possible approaches, such as computing the entropy of each series - the uniform distribution maximizes the entropy, so if the entropy is suspiciously low you would conclude that you probably don't have a uniform distribution. That works as a measure of uniformity in some sense.

Another suggestion would be to use a measure like the Kullback-Leibler divergence, which measures the similarity of two distributions.

  • $\begingroup$ I have a couple of questions regarding your reply: 1. Why do you state that chi-square does not give a measure of uniformity? Isn't a fit test with a uniform distribution a measure of uniformity? 2. How can we know when should we use either chi-square or entropy ? $\endgroup$ Commented Nov 8, 2012 at 5:44
  • $\begingroup$ @kanzen_master: I guess that the chi-squared statistic can be seen as a measure of uniformity, but it has some drawbacks, such as the lack of convergence, dependence on the arbitrarily placed bins, that the number of expected counts in the cells needs to be sufficiently large, etc. Which measure/test to use is a matter of taste though, and entropy is not without its problems either (in particular, there are many different estimators of the entropy of a distribution). To me, entropy seems like a less arbitrary measure and is easier to interpret. $\endgroup$
    – MånsT
    Commented Nov 19, 2012 at 8:28
  • $\begingroup$ Is it really necessary to have the actual counts? Would the test not work on the frequencies directly? $\endgroup$ Commented Jun 24, 2020 at 15:52
  • 1
    $\begingroup$ @MichaelMior: you need the counts to compute the chi-squared statistic. $\endgroup$
    – MånsT
    Commented Jun 25, 2020 at 6:48
  • $\begingroup$ @MånsT But why couldn't the frequencies be used instead? $\endgroup$ Commented Jun 25, 2020 at 14:06

In addition to @MansT 's good ideas, you could come up with other measures, but it depends on what you mean by "non-uniformity". To keep it simple, let's look at 4 levels. Perfect uniformity is easy to define:

25 25 25 25

but which of the following is more non-uniform?

20 20 30 30 or 20 20 25 35

or are they equally non-uniform?

if you think they are equally non-uniform, you could use a measure based on the sum of the absolute values of the deviations from normal, scaled by the maximum possible. Then the first is 5 + 5 + 5 + 5 = 20 and the second is 5 + 5 + 0 + 10 = 20. But if you think the second is more nonuniform, you could use something based on the squared deviations in which case the first gets 25 + 25 + 25 + 25 = 100 and the second gets 25 + 25 + 0 + 100 = 150.

  • 1
    $\begingroup$ You seem to be interpreting "uniformly distributed" as "equal", Peter. Whether that is the OP's intention is a valid point to raise, but really should appear as a comment to the question. $\endgroup$
    – whuber
    Commented Apr 4, 2012 at 12:52
  • 1
    $\begingroup$ Hi @whuber That seemed to be what he meant, from the question. What else might it mean? $\endgroup$
    – Peter Flom
    Commented Apr 5, 2012 at 11:37
  • 3
    $\begingroup$ "Equal" means the CDF is $F(x) = 1$ for $x\ge \mu$, $F(x) = 0$ for $x\lt \mu$ while "uniform" means $F(x) = (x-\alpha)/\theta$ for $x \in [\alpha, \alpha+\theta]$. You define "perfect uniformity" in the first sense whereas the standard statistical sense is the second. $\endgroup$
    – whuber
    Commented Apr 5, 2012 at 14:21
  • 1
    $\begingroup$ @whuber, it seems to me the first thing is closer to what the original poster meant by "uniform". Looking at it again, it seems like he/she was using "uniform" to mean "low variance". $\endgroup$
    – Macro
    Commented Apr 6, 2012 at 12:40

Here is a simple heuristic: if you assume elements in any vector sum to $1$ (or simply normalize each element with the sum to achieve this), then uniformity can be represented by L2 norm, which ranges from $\frac{1}{\sqrt d}$ to $1$, with $d$ being the dimension of vectors.

The lower bound $\frac{1}{\sqrt d}$ corresponds to uniformity and upper bound to the $1$-hot vector.

To scale this to a score between $0$ and $1$, you can use $\frac{n*\sqrt d - 1}{\sqrt d - 1}$, where $n$ is the L2 norm.

An example modified from yours with elements summing to $1$ and all vectors with the same dimension for simplicity:

0.10    0.11    0.10    0.09    0.09    0.11    0.10    0.10    0.12    0.08
0.10    0.10    0.10    0.08    0.12    0.12    0.09    0.09    0.12    0.08
0.03    0.02    0.61    0.02    0.03    0.07    0.06    0.05    0.06    0.05

The following will yield $0.0028$, $0.0051$, and $0.4529$ for the rows:

for i=1:size(m); 
    disp( (norm(m(i,:))*sqrt(d)-1) / (sqrt(d)-1) ); 
  • 1
    $\begingroup$ That works nicely. But why (or under what circumstances) should it be preferred to any other $L_p$ norm or to the other solutions offered on this thread? $\endgroup$
    – whuber
    Commented Mar 31, 2014 at 22:03
  • 1
    $\begingroup$ @whuber that I don't know and I don't know of any research on this. Basically it's something I've used as a heuristic that might fit what OP is after, and I don't really claim it being a preferred approach. $\endgroup$
    – user495285
    Commented Apr 2, 2014 at 22:42
  • $\begingroup$ @whuber - Could you figure out theoretically why this works so well. I need to cite this. $\endgroup$
    – Ketan
    Commented Nov 29, 2016 at 5:56
  • $\begingroup$ @user495285 - This seems to work directly with values, and not just frequencies. In your experience, is it better to use it only with frequencies or is it okay to use it directly on a vector. $\endgroup$
    – Ketan
    Commented Nov 29, 2016 at 6:00
  • 1
    $\begingroup$ This is basically equivalent to computing the standard deviation of the points. If the values are viewed as coordinates of an $d$-dimension point, then the $L_2$ norm is just the distance from the origin (or equivalently, the magnitude of the vector). The squaring of each coordinate is equivalent to computing the variance, and the square root operation on the $L_2$ norm is just like converting the variance to a standard deviation. This heuristic is no better or worst than just computing the standard deviation of each list and taking the lowest. $\endgroup$ Commented Dec 26, 2017 at 6:44

Stumbled upon this recently, and to add to the answer from @user495285, as far as I understand it:

When the values are normalized and sum to one, then the uniform distribution is the unit sphere in $\mathbb{R}^n$, and what is being calculated by using an $L_p$ norm is the deviation from the unit sphere using a distance measure of a given $p$, i.e. deviation from the uniform distribution in $\mathbb{R}^n$ with geometric distance measure $p$.

The $L_2$ norm places higher weight on large deviations from the unit sphere in any given dimension, whereas smaller values of $p$ place less weight on large deviations.

When the underlying distribution is the unit sphere, the numerator equals zero in the following equation: $$\frac{n\sqrt{d} - 1}{\sqrt{d} - 1}$$ where $n$ is the $L_2$ norm and $d$ is the vector length.

I believe that the usefulness of geometric measures applies when each position (dimension) of the space described is assumed to be measured on equivalent scales, e.g. all counts of potentially equal distribution. The same assumptions underlying change of bases like PCA/SVD probably are similar here. But then again I'm no mathematician, so I'll leave that open to the more informed.

  • $\begingroup$ Sounds helpful. Could you please point me some reference, so that I can understand this better? I actually need to cite this. $\endgroup$
    – Ketan
    Commented Nov 29, 2016 at 5:57
  • $\begingroup$ You could cite any linear algebra text that covers the Lp norm; this is a very common subject in geometry: how to calculate a distance between two points in an N-dimensional space. You may not even have to cite it depending on your field. $\endgroup$
    – lakinsm
    Commented Dec 4, 2016 at 3:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.