I'm trying to come up with a metric for measuring non-uniformity of a distribution for an experiment I'm running. I have a random variable that should be uniformly distributed in most cases, and I'd like to be able to identify (and possibly measure the degree of) examples of data sets where the variable is not uniformly distributed within some margin.

An example of three data series each with 10 measurements representing frequency of the occurrence of something I'm measuring might be something like this:

a: [10% 11% 10%  9%  9% 11% 10% 10% 12%  8%]
b: [10% 10% 10%  8% 10% 10%  9%  9% 12%  8%]
c: [ 3%  2% 60%  2%  3%  7%  6%  5%  5%  7%]   <-- non-uniform
d: [98% 97% 99% 98% 98% 96% 99% 96% 99% 98%]

I'd like to be able to distinguish distributions like c from those like a and b, and measure c's deviation from a uniform distribution. Equivalently, if there's a metric for how uniform a distribution is (std. deviation close to zero?), I can perhaps use that to distinguish ones with high variance. However, my data may just have one or two outliers, like the c example above, and am not sure if that will be easily detectable that way.

I can hack something to do this in software, but am looking for statistical methods/approaches to justify this formally. I took a class years ago, but stats is not my area. This seems like something that should have a well-known approach. Sorry if any of this is completely bone-headed. Thanks in advance!


If you have not only the frequencies but the actual counts, you can use a $\chi^2$ goodness-of-fit test for each data series. In particular, you wish to use the test for a discrete uniform distribution. This gives you a good test, which allows you to find out which data series are likely not to have been generated by a uniform distribution, but does not provide a measure of uniformity.

There are other possible approaches, such as computing the entropy of each series - the uniform distribution maximizes the entropy, so if the entropy is suspiciously low you would conclude that you probably don't have a uniform distribution. That works as a measure of uniformity in some sense.

Another suggestion would be to use a measure like the Kullback-Leibler divergence, which measures the similarity of two distributions.

| cite | improve this answer | |
  • $\begingroup$ I have a couple of questions regarding your reply: 1. Why do you state that chi-square does not give a measure of uniformity? Isn't a fit test with a uniform distribution a measure of uniformity? 2. How can we know when should we use either chi-square or entropy ? $\endgroup$ – kanzen_master Nov 8 '12 at 5:44
  • $\begingroup$ @kanzen_master: I guess that the chi-squared statistic can be seen as a measure of uniformity, but it has some drawbacks, such as the lack of convergence, dependence on the arbitrarily placed bins, that the number of expected counts in the cells needs to be sufficiently large, etc. Which measure/test to use is a matter of taste though, and entropy is not without its problems either (in particular, there are many different estimators of the entropy of a distribution). To me, entropy seems like a less arbitrary measure and is easier to interpret. $\endgroup$ – MånsT Nov 19 '12 at 8:28
  • $\begingroup$ Is it really necessary to have the actual counts? Would the test not work on the frequencies directly? $\endgroup$ – Michael Mior Jun 24 at 15:52
  • $\begingroup$ @MichaelMior: you need the counts to compute the chi-squared statistic. $\endgroup$ – MånsT Jun 25 at 6:48
  • $\begingroup$ @MånsT But why couldn't the frequencies be used instead? $\endgroup$ – Michael Mior Jun 25 at 14:06

In addition to @MansT 's good ideas, you could come up with other measures, but it depends on what you mean by "non-uniformity". To keep it simple, let's look at 4 levels. Perfect uniformity is easy to define:

25 25 25 25

but which of the following is more non-uniform?

20 20 30 30 or 20 20 25 35

or are they equally non-uniform?

if you think they are equally non-uniform, you could use a measure based on the sum of the absolute values of the deviations from normal, scaled by the maximum possible. Then the first is 5 + 5 + 5 + 5 = 20 and the second is 5 + 5 + 0 + 10 = 20. But if you think the second is more nonuniform, you could use something based on the squared deviations in which case the first gets 25 + 25 + 25 + 25 = 100 and the second gets 25 + 25 + 0 + 100 = 150.

| cite | improve this answer | |
  • 1
    $\begingroup$ You seem to be interpreting "uniformly distributed" as "equal", Peter. Whether that is the OP's intention is a valid point to raise, but really should appear as a comment to the question. $\endgroup$ – whuber Apr 4 '12 at 12:52
  • $\begingroup$ Hi @whuber That seemed to be what he meant, from the question. What else might it mean? $\endgroup$ – Peter Flom Apr 5 '12 at 11:37
  • 2
    $\begingroup$ "Equal" means the CDF is $F(x) = 1$ for $x\ge \mu$, $F(x) = 0$ for $x\lt \mu$ while "uniform" means $F(x) = (x-\alpha)/\theta$ for $x \in [\alpha, \alpha+\theta]$. You define "perfect uniformity" in the first sense whereas the standard statistical sense is the second. $\endgroup$ – whuber Apr 5 '12 at 14:21
  • $\begingroup$ @whuber, it seems to me the first thing is closer to what the original poster meant by "uniform". Looking at it again, it seems like he/she was using "uniform" to mean "low variance". $\endgroup$ – Macro Apr 6 '12 at 12:40
  • $\begingroup$ That's just it, Macro: we cannot really say. The question needs clarification before it deserves an answer, IMHO. The accepted answer suggests the OP used "uniform" in the standard statistical sense. $\endgroup$ – whuber Apr 6 '12 at 15:52

Here is a simple heuristic: if you assume elements in any vector sum to $1$ (or simply normalize each element with the sum to achieve this), then uniformity can be represented by L2 norm, which ranges from $\frac{1}{\sqrt d}$ to $1$, with $d$ being the dimension of vectors.

The lower bound $\frac{1}{\sqrt d}$ corresponds to uniformity and upper bound to the $1$-hot vector.

To scale this to a score between $0$ and $1$, you can use $\frac{n*\sqrt d - 1}{\sqrt d - 1}$, where $n$ is the L2 norm.

An example modified from yours with elements summing to $1$ and all vectors with the same dimension for simplicity:

0.10    0.11    0.10    0.09    0.09    0.11    0.10    0.10    0.12    0.08
0.10    0.10    0.10    0.08    0.12    0.12    0.09    0.09    0.12    0.08
0.03    0.02    0.61    0.02    0.03    0.07    0.06    0.05    0.06    0.05

The following will yield $0.0028$, $0.0051$, and $0.4529$ for the rows:

for i=1:size(m); 
    disp( (norm(m(i,:))*sqrt(d)-1) / (sqrt(d)-1) ); 
| cite | improve this answer | |
  • 1
    $\begingroup$ That works nicely. But why (or under what circumstances) should it be preferred to any other $L_p$ norm or to the other solutions offered on this thread? $\endgroup$ – whuber Mar 31 '14 at 22:03
  • $\begingroup$ @whuber that I don't know and I don't know of any research on this. Basically it's something I've used as a heuristic that might fit what OP is after, and I don't really claim it being a preferred approach. $\endgroup$ – user495285 Apr 2 '14 at 22:42
  • $\begingroup$ @whuber - Could you figure out theoretically why this works so well. I need to cite this. $\endgroup$ – Ketan Nov 29 '16 at 5:56
  • $\begingroup$ @user495285 - This seems to work directly with values, and not just frequencies. In your experience, is it better to use it only with frequencies or is it okay to use it directly on a vector. $\endgroup$ – Ketan Nov 29 '16 at 6:00
  • $\begingroup$ @Ketan I would advise against citing this post, because it's heuristic and theoretically unsupported. (Because it does not rely on any consideration of sampling variation, there is no way to tell--without further analysis--whether it's any good.) It just so happens, though, that the $L_2$ norm has a simple algebraic connection to the $\chi^2$ statistic used in goodness of fit tests: that's the reason it might be suitable to measure non-uniformity. $\endgroup$ – whuber Nov 29 '16 at 18:58

Stumbled upon this recently, and to add to the answer from @user495285, as far as I understand it:

When the values are normalized and sum to one, then the uniform distribution is the unit sphere in $\mathbb{R}^n$, and what is being calculated by using an $L_p$ norm is the deviation from the unit sphere using a distance measure of a given $p$, i.e. deviation from the uniform distribution in $\mathbb{R}^n$ with geometric distance measure $p$.

The $L_2$ norm places higher weight on large deviations from the unit sphere in any given dimension, whereas smaller values of $p$ place less weight on large deviations.

When the underlying distribution is the unit sphere, the numerator equals zero in the following equation: $$\frac{n\sqrt{d} - 1}{\sqrt{d} - 1}$$ where $n$ is the $L_2$ norm and $d$ is the vector length.

I believe that the usefulness of geometric measures applies when each position (dimension) of the space described is assumed to be measured on equivalent scales, e.g. all counts of potentially equal distribution. The same assumptions underlying change of bases like PCA/SVD probably are similar here. But then again I'm no mathematician, so I'll leave that open to the more informed.

| cite | improve this answer | |
  • $\begingroup$ Sounds helpful. Could you please point me some reference, so that I can understand this better? I actually need to cite this. $\endgroup$ – Ketan Nov 29 '16 at 5:57
  • $\begingroup$ You could cite any linear algebra text that covers the Lp norm; this is a very common subject in geometry: how to calculate a distance between two points in an N-dimensional space. You may not even have to cite it depending on your field. $\endgroup$ – lakinsm Dec 4 '16 at 3:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.