Question regarding asymptotic assumptions and hypotesis testing paradox for large samples

Question

Suppose we would like to verify if a r.v $X$ follows a distribution with cumulative density as $F$, if $n$ goes to $\infty$ I'm able to use komogorov test which states reject $H_0$ (stating that $X$ does follow the such density) when:

$\sqrt{n}.sup|F(x)-F_{emp}(x)|>K_{\alpha}$

where $K_{\alpha}$ is some quantile from komogorov distribution and $F_{emp}$ as empiric cumulative distribution

But wikipedia states that:

That's komogorov test will ever reject $H_0$ when $n \rightarrow \infty$ as $power=P(Reject \hspace{0.1cm} H_0|\theta)$ where $\theta $ is the 'maximum' (supreme) distance between empiric cumulative and $F$.

So here's my question:

Why is this test even useful? since it'll reject $H_0$ when $n\rightarrow \infty$ and the test holds only if $n \rightarrow \infty$.

PS: It's a specific instance from this problem there is so many more tests which lead this paradox.

Answer 1

My understanding of the K-S and other goodness-of-fit tests is that the typical null hypothesis states the data does follow a particular distribution of interest. Unless the data generative process truly follows the theoretical model specified, the null hypothesis will be rejected almost surely with increasing sample size.

Perhaps you are suggesting that no real data generative process actually follows a theoretical model so we might as well just reject any goodness-of-fit null hypothesis without bothering to do a test. In practice I generally do not perform these goodness-of-fit tests when deciding on a probability model. To me a model is a convenient, perhaps imperfect, representation of the data generative process. If one has an exorbitant sample size and an empirical distribution estimate like a histogram clearly shows lack of fit, then the KS test would just be a formal way of reporting the "eyeball test."