Suppose we would like to verify if a r.v $X$ follows a distribution with cumulative density as $F$, if $n$ goes to $\infty$ I'm able to use komogorov test which states reject $H_0$ (stating that $X$ does follow the such density) when:


where $K_{\alpha}$ is some quantile from komogorov distribution and $F_{emp}$ as empiric cumulative distribution

But wikipedia states that:

enter image description here

That's komogorov test will ever reject $H_0$ when $n \rightarrow \infty$ as $power=P(Reject \hspace{0.1cm} H_0|\theta)$ where $\theta $ is the 'maximum' (supreme) distance between empiric cumulative and $F$.

So here's my question:

Why is this test even useful? since it'll reject $H_0$ when $n\rightarrow \infty$ and the test holds only if $n \rightarrow \infty$.

PS: It's a specific instance from this problem there is so many more tests which lead this paradox.


1 Answer 1


My understanding of the K-S and other goodness-of-fit tests is that the typical null hypothesis states the data does follow a particular distribution of interest. Unless the data generative process truly follows the theoretical model specified, the null hypothesis will be rejected almost surely with increasing sample size.

Perhaps you are suggesting that no real data generative process actually follows a theoretical model so we might as well just reject any goodness-of-fit null hypothesis without bothering to do a test. In practice I generally do not perform these goodness-of-fit tests when deciding on a probability model. To me a model is a convenient, perhaps imperfect, representation of the data generative process. If one has an exorbitant sample size and an empirical distribution estimate like a histogram clearly shows lack of fit, then the KS test would just be a formal way of reporting the "eyeball test."

  • $\begingroup$ sorry you're sure but the problem remains. $\endgroup$ Dec 6, 2021 at 23:00
  • $\begingroup$ It seemed you stated the null hypothesis is that the data does not follow a specific theoretical distribution. If that is not what you intended to write, I'm not sure I understand your concern. The test does not require an impossibly infinite sample size in order to "hold." The test is constructed to have a type I error rate if the null is true and it has a type II error rate if the alternative is true. $\endgroup$ Dec 7, 2021 at 1:35
  • $\begingroup$ Having high power when the alternative is true is not a bad thing. A proper feature of every hypothesis test is that the power under the alternative tends to 1 with increasing sample size. Consider a Wald confidence interval for a sample proportion. With increasing sample size the width of this interval goes to zero so that the power to reject a false hypothesis tends to 1. $\endgroup$ Dec 7, 2021 at 1:37
  • $\begingroup$ The concern is the test just works if $n\rightarrow \infty$ but when it occurs so power=1, that's I'll always reject $H_0$ whatever cumulative I'm comparing then what is the test for? $\endgroup$ Dec 7, 2021 at 2:03
  • 1
    $\begingroup$ I wouldn't say it is utterly useless, but it is certainly not needed as a routine course of practice. I make my model fitting decisions by visual inspection. If I need a metric to justify my model selection to someone else then I will use AIC. Rather than "rejecting" or "accepting" a hypothesis with the KS test one could use the p-value from a KS test as a weight of the evidence when choosing between probability models, analogous to AIC. $\endgroup$ Dec 8, 2021 at 0:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.