# Chi2 test on big uniformly random sample [duplicate]

I expect that, in Python with numpy and scipy,

scipy.stats.chisquare(numpy.bincount(numpy.random.randint(100, size=1000000)))


will return P-value which is very close to 1.

In reality, I constantly get different P-values which are not close to 1 at all. P-values of different runs are scattered from 0 to 1, no matter how many categories I have and how big sample is.

It seems I don't understand something very basic. But what? Is chi2 test applicable here?

If not, how can I test uniformity of pseudo-random values generated by machine? (I know that there are a lot of tests for random number generators. I'm looking for test for uniformity.)

# What I didn't understand

Indeed, P-value is distributed uniformly if null-hypothesis is true. Looks obvious when I understood it.

My mistake was that I expected to have P-value to be close to 1. If distribution is really uniform, P-value should be uniformly scattered over [0, 1]. And I should check that P-value greater than some value close to 0. If distribution weren't uniform, I would get values close to 0, since there is very low (close to 0) probability that I would get more extreme results if distribution were uniform.

• @whuber Thank you for pointing to this question. It doesn't directly address my misunderstanding but prompted me to rethink and come with answer on my own. Jul 26, 2016 at 15:52
• I believe it does directly address the misunderstanding reflected in your first line, where you state you expect the p-value to be close to $1$. It shows instead that you should expect the p-value to behave like a uniform random variable between $0$ and $1$ and explains why--which also addresses your second observation about how you get different p-values.
– whuber
Jul 26, 2016 at 16:07
• @whuber Yes, you're right. I saw your comment just after I updated my question. Looks so obvious now... Jul 26, 2016 at 16:09