Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I'm trying to come up with a metric for measuring non-uniformity of a distribution for an experiment I'm running. I have a random variable that should be uniformly distributed in most cases, and I'd like to be able to identify (and possibly measure the degree of) examples of data sets where the variable is not uniformly distributed within some margin.

An example of three data series each with 10 measurements representing frequency of the occurrence of something I'm measuring might be something like this:

a: [10% 11% 10% 9% 9% 11% 10% 10% 12% 8%]
b: [10% 10% 10% 8% 10% 10% 9%  9% 12% 8%]
c: [3% 2%  60%  2%  3% 7% 6% 5% 6% 5% 7%]   <-- non-uniform

I'd like to be able to distinguish distributions like c from those like a and b, and measure c's deviation from a uniform distribution. Equivalently, if there's a metric for how uniform a distribution is (std. deviation close to zero??), I can probably use that to distinguish ones with high variance? But, my data may just have one or two outliers, like the c example above, and am not sure if that will be easily detectable that way.

I can hack something to do this in software, but am looking for statistical methods/approaches to justify this formally. I took a class years ago, but stats is not my area, but this seems like something that should have a well-known approach. Sorry if any of this is completely bone-headed. Thanks in advance!

share|improve this question
add comment

3 Answers 3

up vote 10 down vote accepted

If you have not only the frequencies but the actual counts, you can use a $\chi^2$ goodness-of-fit test for each data series. In particular, you wish to use the test for a discrete uniform distribution. This gives you a good test, which allows you to find out which data series are likely not to have been generated by a uniform distribution, but does not provide a measure of uniformity.

There are other possible approaches, such as computing the entropy of each series - the uniform distribution maximizes the entropy, so if the entropy is suspiciously low you would conclude that you probably don't have a uniform distribution. That works as a measure of uniformity in some sense.

Another suggestion would be to use a measure like the Kullback-Leibler divergence, which measures the similarity of two distributions.

share|improve this answer
    
I have a couple of questions regarding your reply: 1. Why do you state that chi-square does not give a measure of uniformity? Isn't a fit test with a uniform distribution a measure of uniformity? 2. How can we know when should we use either chi-square or entropy ? –  kanzen_master Nov 8 '12 at 5:44
    
@kanzen_master: I guess that the chi-squared statistic can be seen as a measure of uniformity, but it has some drawbacks, such as the lack of convergence, dependence on the arbitrarily placed bins, that the number of expected counts in the cells needs to be sufficiently large, etc. Which measure/test to use is a matter of taste though, and entropy is not without its problems either (in particular, there are many different estimators of the entropy of a distribution). To me, entropy seems like a less arbitrary measure and is easier to interpret. –  MånsT Nov 19 '12 at 8:28
add comment

In addition to @MansT 's good ideas, you could come up with other measures, but it depends on what you mean by "non-uniformity". To keep it simple, let's look at 4 levels. Perfect uniformity is easy to define:

25 25 25 25

but which of the following is more non-uniform?

20 20 30 30 or 20 20 25 35

or are they equally non-uniform?

if you think they are equally non-uniform, you could use a measure based on the sum of the absolute values of the deviations from normal, scaled by the maximum possible. Then the first is 5 + 5 + 5 + 5 = 20 and the second is 5 + 5 + 0 + 10 = 20. But if you think the second is more nonuniform, you could use something based on the squared deviations in which case the first gets 25 + 25 + 25 + 25 = 100 and the second gets 25 + 25 + 0 + 100 = 150.

share|improve this answer
1  
You seem to be interpreting "uniformly distributed" as "equal", Peter. Whether that is the OP's intention is a valid point to raise, but really should appear as a comment to the question. –  whuber Apr 4 '12 at 12:52
    
Hi @whuber That seemed to be what he meant, from the question. What else might it mean? –  Peter Flom Apr 5 '12 at 11:37
2  
"Equal" means the CDF is $F(x) = 1$ for $x\ge \mu$, $F(x) = 0$ for $x\lt \mu$ while "uniform" means $F(x) = (x-\alpha)/\theta$ for $x \in [\alpha, \alpha+\theta]$. You define "perfect uniformity" in the first sense whereas the standard statistical sense is the second. –  whuber Apr 5 '12 at 14:21
    
@whuber, it seems to me the first thing is closer to what the original poster meant by "uniform". Looking at it again, it seems like he/she was using "uniform" to mean "low variance". –  Macro Apr 6 '12 at 12:40
    
That's just it, Macro: we cannot really say. The question needs clarification before it deserves an answer, IMHO. The accepted answer suggests the OP used "uniform" in the standard statistical sense. –  whuber Apr 6 '12 at 15:52
add comment

Here is a simple heuristic: if you assume elements in any vector sum to $1$ (or simply normalize each element with the sum to achieve this), then uniformity can be represented by L2 norm, which ranges from $1/\sqrt d$ to $1$, with $d$ being the dimension of vectors. The lower bound $1/\sqrt d$ corresponds to uniformity and upper bound to the $1$-hot vector.

To scale this to a score between $0$ and $1$, you can use $(n*\sqrt d - 1)/(\sqrt d - 1)$, where $n$ is the L2 norm.

An example modified from urs --- with elements summing to $1$ and all vectors with the same dimension for simplicity:

0.10    0.11    0.10    0.09    0.09    0.11    0.10    0.10    0.12    0.08
0.10    0.10    0.10    0.08    0.12    0.12    0.09    0.09    0.12    0.08
0.03    0.02    0.61    0.02    0.03    0.07    0.06    0.05    0.06    0.05

The following will yield $0.0028$, $0.0051$, and $0.4529$ for the rows:

d=size(m,2); 
for i=1:size(m); 
    disp( (norm(m(i,:))*sqrt(d)-1) / (sqrt(d)-1) ); 
end
share|improve this answer
    
That works nicely. But why (or under what circumstances) should it be preferred to any other $L_p$ norm or to the other solutions offered on this thread? –  whuber Mar 31 at 22:03
    
@whuber that I don't know and I don't know of any research on this. Basically it's something I've used as a heuristic that might fit what OP is after, and I don't really claim it being a preferred approach. –  user495285 Apr 2 at 22:42
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.