# Distribution of p-values from scipy ks-test

I wast trying to understand how kstest works using scipy. I generated chi2 random variables and then checked if the values follow the chi2 distribution. The codes are shown below.

from scipy import stats
pl = []
for i in range(1000):
pl.append(stats.kstest(stats.chi2(1).rvs(size=20000), stats.chi2(1).cdf))
plt.hist(pl[1], bins=20)
plt.show()


I expected that the p value would be distributed continuously between 0 and 1. But as you can see in the figure, this is not the case.

Is something wrong with my test codes, or could you please explain why the p-value distribution is not uniform nor continuous?

• Your histogram indicates there are only two data points. This is likely because when you ask to plot the histogram for pl[1], you are actually plotting the tuple returned from the second call to stats.kstest in the loop (which is the test statistic and p-value), rather than all 1000 p-values. – B.Liu Jan 19 at 0:13
• @B.Liu You're right! Now the distribution looks uniform. When write this comment as an answer, then I would select that. Thank you. – Nownuri Jan 19 at 1:16

(Expanded from initial comment) Your histogram indicates there are only two data points.

This is because when you plot the histogram for pl[1], you are actually plotting the tuple returned from the second call to stats.kstest() in the for loop (which contains the test statistic and p-value of the second K-S test), rather than all 1000 p-values.

To plot all 1000 p-values, you can modify the code snippet to the following:

from scipy import stats
from matplotlib import pyplot as plt

pl = []
for i in range(1000):
pl.append(stats.kstest(stats.chi2(1).rvs(size=20000), stats.chi2(1).cdf)[1])
plt.hist(pl, bins=20)
plt.show()


which moves the [1] from the plt.hist line to the pl.append line.

On my first run of the code above I obtained the following histogram:

... which demonstrates a sort-of uniform shape as expected.