2
$\begingroup$

I expect that, in Python with numpy and scipy,

scipy.stats.chisquare(numpy.bincount(numpy.random.randint(100, size=1000000)))

will return P-value which is very close to 1.

In reality, I constantly get different P-values which are not close to 1 at all. P-values of different runs are scattered from 0 to 1, no matter how many categories I have and how big sample is.

It seems I don't understand something very basic. But what? Is chi2 test applicable here?

If not, how can I test uniformity of pseudo-random values generated by machine? (I know that there are a lot of tests for random number generators. I'm looking for test for uniformity.)

What I didn't understand

Indeed, P-value is distributed uniformly if null-hypothesis is true. Looks obvious when I understood it.

My mistake was that I expected to have P-value to be close to 1. If distribution is really uniform, P-value should be uniformly scattered over [0, 1]. And I should check that P-value greater than some value close to 0. If distribution weren't uniform, I would get values close to 0, since there is very low (close to 0) probability that I would get more extreme results if distribution were uniform.

Improve this question
$\endgroup$
3
  • $\begingroup$ @whuber Thank you for pointing to this question. It doesn't directly address my misunderstanding but prompted me to rethink and come with answer on my own. $\endgroup$
    – George Sovetov
    Jul 26, 2016 at 15:52
  • 1
    $\begingroup$ I believe it does directly address the misunderstanding reflected in your first line, where you state you expect the p-value to be close to $1$. It shows instead that you should expect the p-value to behave like a uniform random variable between $0$ and $1$ and explains why--which also addresses your second observation about how you get different p-values. $\endgroup$
    – whuber ♦
    Jul 26, 2016 at 16:07
  • 1
    $\begingroup$ @whuber Yes, you're right. I saw your comment just after I updated my question. Looks so obvious now... $\endgroup$
    – George Sovetov
    Jul 26, 2016 at 16:09

0

Reset to default