Apologies in advance if I am thinking about this in the totally wrong way. Essentially, I have a theoretical distribution that I want to test, using the KS test, if my numerical data follows the same distribution. I know that it should follow the same distribution, and get an accordingly very low D statistic. At p=0.05 for example, I clearly cannot reject the null hypothesis, however, I am confused whether this strongly supports the fact that my null hypothesis is correct. Would I need to use a high p-value instead to support the null hypothesis that they are the same distribution?

Again, I am quite confused about this so I apologise if it does not make sense.



1 Answer 1


Numerous posts on site address this issue, in various guises.

however, I am confused whether this strongly supports the fact that my null hypothesis is correct.

It can't! Let us assume the null is true for a moment. Nevertheless there's essentially always going to be population distributions that are closer to the data than the hypothesized (i.e. actual) distribution is.

This is no different from the problem of demonstrating that the population mean is some hypothesised value.

Indeed even if the p value was exactly 1 you still could not assert the null; there's an infinite number of adjacent alternatives with p value as close as you like to 1.

Data cannot demonstrate an equality null; you can sometimes discover a discrepancy large enough to place doubt on it.

You might consider whether an equivalence test might make more sense for your circumstances, but if not there's generally going to be little you can do.

  • $\begingroup$ Thank you for your answer, makes a lot more sense now! So would there be any benefit in showing that the null hypothesis cannot be rejected for large p? $\endgroup$
    – H_Boofer
    Mar 28, 2022 at 2:50
  • 1
    $\begingroup$ You can certainly show that the data are not inconsistent with an equality null, but that doesn't exclude all those alternatives I discussed; saying the null is plausible is a pretty weak statement. However, it would be a rare situation indeed where you can really assert some model to be exactly correct. Not even in physics: Einstein replaced Newton, for example, after hundreds of years, but even general relativity is not consistent with quantum theory, so it looks like that, too, will eventually give way to something else. $\endgroup$
    – Glen_b
    Mar 28, 2022 at 2:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.