2
$\begingroup$

I am working on a few algorithms where I have a list of $N$ samples. Currently I have plotted these into a histogram and have a view of how uniform the values are distributed within an interval, which is quite good as a visualization, although I need a comparable value of how uniform the dataset is, in order to measure how robust it is compared to my other algorithms.

I have been looking at chi-squared test, but could not figure out how it would become helpful in my usecase?

Sample from dataset:

8725
462
1492
972
9941
8235
8220
6949
1252

Code for importing data and applying chi-squared in R:

mydata = read.csv2("/opt/doc/stat/uniform_test_1.csv")
x <- sapply(mydata, as.numeric)
chisq.test(x)

Result: X-squared = 1664769844, df = 999998, p-value < 2.2e-16

Improve this question
$\endgroup$
  • $\begingroup$ You are asking a (simplified) one-dimensional version of stats.stackexchange.com/questions/40928/…, all of whose answers are applicable to your question with essentially no change at all. $\endgroup$ – whuber♦ Jan 8 '13 at 18:27
  • $\begingroup$ @whuber, thank you for pointing me to that question. But i must admit that this part of statistics is a bit rusty in my mind, can you be a bit more specific? Thanks. $\endgroup$ – JavaCake Jan 8 '13 at 18:40
  • 1
    $\begingroup$ A Baumann suggests a KS-test: read about it on Wikipedia and find an open-source implementation in R. Ben Allison recommends a chi-squared test and helpfully describes how to conduct it by dividing the range of data "into equally sized non-overlapping patches." Many sources, including Wikipedia, describe this test and provide formulas. Open source software such as R will do the calculations automatically. $\endgroup$ – whuber♦ Jan 8 '13 at 18:44
  • $\begingroup$ @whuber, i have tried to apply chisq.test to my dataset and get following result X-squared = 1664769844, df = 999998, p-value < 2.2e-16. What does the df represent? I will look into KS-test. $\endgroup$ – JavaCake Jan 8 '13 at 18:51
  • $\begingroup$ You may have misapplied it (although it's conceivable this result is correct): in an edit to your question, can you provide some details describing what you did? df is the D egrees of F reedom mentioned in all the references. $\endgroup$ – whuber♦ Jan 8 '13 at 18:52
3
$\begingroup$

Chi-squared is used in a LOT of ways in statistics. The R command chisq.test is described as: "chisq.test performs chi-squared contingency table tests and goodness-of-fit tests." And in particular, "If x ... is a vector and y is not given, then a goodness-of-fit test is performed (x is treated as a one-dimensional contingency table)." So if your $x$ is your raw data, you're getting nonsensical results.

It sounds like you're conflicted on what you're calling "uniform". Visually, you're looking at a histogram, which bins the data in intervals and displays the counts in each interval. Yet you don't require numbers to be equally divided in your interval?

Based on what you're seeing in the histogram, you should bin your data, as in the histogram's bins, and then you can do a chisq.test on that, or look at the variance among the bins, or look at quantiles of the bins, or something else.

From what you've said, the big difference between what you want and checking a random number generator is that you don't care about the order in which the numbers were generated, only the set of numbers that were generated. In which case, you'd expect the count of numbers in each bin to be proportional to the size of the bins, and deviance from that would indicate non-uniformity.

Improve this answer
$\endgroup$
  • $\begingroup$ I have realised that my approach to uniform values perhaps has been false. You mention binning my data, can you please explain how i should input the data to chisq.test? The doc stat.ethz.ch/R-manual/R-patched/library/stats/html/… does not explain in detail how the data should be. $\endgroup$ – JavaCake Jan 8 '13 at 21:22
  • $\begingroup$ @JavaCake: There are various means for binning data. If you're using the hist function to generate your histograms, it does more than just graph, and you can do something like: foo <- hist (x) and then chisq.test (foo$counts). That is, the count of how many numbers fall into each bin -- indicated by a bar in the graph -- is what you analyze. The hist command may not create as many bins as you need, and it only works on the data you present so it doesn't know if there really should be bins to the left or right of your existing data. $\endgroup$ – Wayne Jan 8 '13 at 21:31
  • $\begingroup$ That gave infact a result. Since my interval is $[1:10^4]$ would the chisq function need this information in order to give the correct result? $\endgroup$ – JavaCake Jan 8 '13 at 21:39
  • $\begingroup$ My idea was to set breaks=max for hist and overcome that problem with not getting all bins? $\endgroup$ – JavaCake Jan 8 '13 at 21:46
  • $\begingroup$ @JavaCake: I'd look at cut and table instead of leaning on hist, but you need to tell hist what your actual range is. Something like: foo <- hist (x, breaks=seq (1, 10000, length.out=10)) Which says your data ranges from 1 to 10000 and it should be broken up into 10 bins. You should look at the documentation for seq (type ?seq) for more details. $\endgroup$ – Wayne Jan 8 '13 at 21:50
1
$\begingroup$

I think if you're after a measure of uniformity, goodness of fit tests for the uniform offer a variety of statistics that can provide suitable 'uniformity' measures.

If your upper and lower limits are known, Kolomogorov-Smirnov, Cramer-von Mises or Anderson-Darling statistics offer measures of uniformity (though there are a bunch of other measures available from other statistics).

If the upper and lower limits are unknown, you could do correlation against uniform scores (expected uniform order statistics or similar), which doesn't depend on the limits being known.

An alternative is to use the sample max and min to scale the remainder of the sample to $(0,1)$; if the sample is from a uniform that rescaling leaves you with a standard uniform sample with two fewer observations; then one of the goodness-of-fit statistics can be used as a measure of uniformity.

Chi-square test statistics can be used but they don't make efficient use of the available information (they have relatively low power against interesting alternatives).

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.