1
$\begingroup$

Right now I'm generating 100 Beta RV's and using these data to test the null hypothesis of whether they are uniform, but my Kolmogorov-Smirnov p-value fluctuates greatly (from far below 0.05 to far above 0.1) every time I run my code. Is this because 100 RV's are too small for a consistent answer?

$\endgroup$
5
$\begingroup$

It is usual for P-values to fluctuate because P-values are dependent on the data. Every dataset will provide different levels of evidence against the null hypothesis and so the P-values will differ.

If the null hypothesis is true then the P-value should vary as a random variate with a uniform distribution between 0 and 1. If the null is false then the P-value will still vary, but it will tend to be smaller with increasing disparity between the 'true' state and the null hypothesis, and with larger sample. Yes, a smaller sample will yield more variability of P-value for any given disparity.

It is convenient to think of P-values as a data-dependent random variable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.