Defining the “uniformity” of a dataset

Question

I am working on a few algorithms where I have a list of $N$ samples. Currently I have plotted these into a histogram and have a view of how uniform the values are distributed within an interval, which is quite good as a visualization, although I need a comparable value of how uniform the dataset is, in order to measure how robust it is compared to my other algorithms.

I have been looking at chi-squared test, but could not figure out how it would become helpful in my usecase?

Sample from dataset:

8725
462
1492
972
9941
8235
8220
6949
1252

Code for importing data and applying chi-squared in R:

mydata = read.csv2("/opt/doc/stat/uniform_test_1.csv")
x <- sapply(mydata, as.numeric)
chisq.test(x)

Result: X-squared = 1664769844, df = 999998, p-value < 2.2e-16

Answer 1

Chi-squared is used in a LOT of ways in statistics. The R command chisq.test is described as: "chisq.test performs chi-squared contingency table tests and goodness-of-fit tests." And in particular, "If x ... is a vector and y is not given, then a goodness-of-fit test is performed (x is treated as a one-dimensional contingency table)." So if your $x$ is your raw data, you're getting nonsensical results.

It sounds like you're conflicted on what you're calling "uniform". Visually, you're looking at a histogram, which bins the data in intervals and displays the counts in each interval. Yet you don't require numbers to be equally divided in your interval?

Based on what you're seeing in the histogram, you should bin your data, as in the histogram's bins, and then you can do a chisq.test on that, or look at the variance among the bins, or look at quantiles of the bins, or something else.

From what you've said, the big difference between what you want and checking a random number generator is that you don't care about the order in which the numbers were generated, only the set of numbers that were generated. In which case, you'd expect the count of numbers in each bin to be proportional to the size of the bins, and deviance from that would indicate non-uniformity.

Answer 2

I think if you're after a measure of uniformity, goodness of fit tests for the uniform offer a variety of statistics that can provide suitable 'uniformity' measures.

If your upper and lower limits are known, Kolomogorov-Smirnov, Cramer-von Mises or Anderson-Darling statistics offer measures of uniformity (though there are a bunch of other measures available from other statistics).

If the upper and lower limits are unknown, you could do correlation against uniform scores (expected uniform order statistics or similar), which doesn't depend on the limits being known.

An alternative is to use the sample max and min to scale the remainder of the sample to $(0,1)$; if the sample is from a uniform that rescaling leaves you with a standard uniform sample with two fewer observations; then one of the goodness-of-fit statistics can be used as a measure of uniformity.

Chi-square test statistics can be used but they don't make efficient use of the available information (they have relatively low power against interesting alternatives).