# Understanding scipy Kolmogorov-Smirnov test [duplicate]

I'm trying to understand the Kolmogorov-Smirnov test using a very simple example. I generate a set of random, uniform values between 0 and 1.0. I then test that these values are from a uniform distribution by using the scipy kstest function. I'm expecing a very small D value and a pvalue close to 1.0, but instead I get wildly varying pvalues every time I run the code. What am I missing?

import numpy as np
import scipy
a = np.random.uniform(size=4999)
print(scipy.stats.kstest(a, 'uniform'))


Here are the outputs of a few consecutive runs:

(0.0075523161200627964, 0.93798952050647577)
(0.013787195268362473, 0.29799260741344774)
(0.014359046616557847, 0.25402403230845855)
(0.012521820948675988, 0.41329007558099806)
(0.011159003477582918, 0.56216895575676396)


## marked as duplicate by whuber♦Aug 27 '14 at 20:14

For the KS test the p-value is itself distributed uniformly in [0,1] if the H0 is true (which it is if you test whether it your sample is from $U(0,1)$ and the random number generation works okay). It therefore must "vary wildly" between 0 and 1, in fact its standard deviation is $1/\sqrt{12}$ which is roughly 0.3.
You can check this by looking whether the percentages of p values smaller or equal to some $p_0$ over your independent consecutive runs is close to said $p_0$.
• Is there any advantage of choosing small $p_0$, like 1e-3 vs 0.5? – Ahmed Fasih Apr 24 '17 at 3:07
• @ahmed No, in what I wrote, there is no significance to the value of $p_0$ at all. The percentages of p values smaller or equal to some $p_0$ should be close to $p_0$ regardless of the concrete $p_0$. That said, for very small $p_0$ one can encounter numeric problems when calculating the p-values. – Momo May 11 '17 at 15:33