0
$\begingroup$

I wast trying to understand how kstest works using scipy. I generated chi2 random variables and then checked if the values follow the chi2 distribution. The codes are shown below.

from scipy import stats 
pl = []
for i in range(1000):
    pl.append(stats.kstest(stats.chi2(1).rvs(size=20000), stats.chi2(1).cdf))
plt.hist(pl[1], bins=20)
plt.show()

I expected that the p value would be distributed continuously between 0 and 1. But as you can see in the figure, this is not the case. The distribution of p-values

Is something wrong with my test codes, or could you please explain why the p-value distribution is not uniform nor continuous?

$\endgroup$
2
  • 3
    $\begingroup$ Your histogram indicates there are only two data points. This is likely because when you ask to plot the histogram for pl[1], you are actually plotting the tuple returned from the second call to stats.kstest in the loop (which is the test statistic and p-value), rather than all 1000 p-values. $\endgroup$ – B.Liu Jan 19 at 0:13
  • $\begingroup$ @B.Liu You're right! Now the distribution looks uniform. When write this comment as an answer, then I would select that. Thank you. $\endgroup$ – Nownuri Jan 19 at 1:16
1
$\begingroup$

(Expanded from initial comment) Your histogram indicates there are only two data points.

This is because when you plot the histogram for pl[1], you are actually plotting the tuple returned from the second call to stats.kstest() in the for loop (which contains the test statistic and p-value of the second K-S test), rather than all 1000 p-values.

To plot all 1000 p-values, you can modify the code snippet to the following:

from scipy import stats 
from matplotlib import pyplot as plt

pl = []
for i in range(1000):
    pl.append(stats.kstest(stats.chi2(1).rvs(size=20000), stats.chi2(1).cdf)[1])
plt.hist(pl, bins=20)
plt.show()

which moves the [1] from the plt.hist line to the pl.append line.

On my first run of the code above I obtained the following histogram: Histogram of the p-value from 1000 K-S test between the theoretical chi-squared distribution and its samples. It demonstrates a uniform shape.

... which demonstrates a sort-of uniform shape as expected.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.