Is the following conclusion correct?

Assume I have two data samples, with $n$ and $m$ the sizes of first and second samples respectively. Next, I perform Kolmogorov-Smirnov test with some significance level $\alpha$. We reject the null-hypothesis if $$ D_{n,m} > c(\alpha)\sqrt{\frac{n+m}{n\cdot m}} $$ where $c$ depends only on the level $\alpha$ and $D_{n,m}$ is the Kolmogorov–Smirnov statistic $$ D_{n,m} = \sup_{x}|F_{1,n}(x) - F_{2,m}(x)|. $$

Assume, that $m$, the size of the second sample, is fixed and I can increase $n$. Note, we fixe only size, but not the sample itself. Then, according to the definition of the power the statistical power of the test goes to 1 as $n\to \infty$ only if $$ \lim_{n\to \infty} \mathbb{P}[D_{n,m} > c(\alpha)/\sqrt{m}|H_{0} \text{ is wrong}] \to 1, $$ i.e. if the size of the second sample is fixed, then, in general, the test is not asymptotically powerful.

Is this alway the case in two-sample tests that both samples sizes must be increasing, possibly with some rate.

  • $\begingroup$ Because $\lim_{n\to\infty}\sqrt{(n+m)/(nm)}=\sqrt{1/m},$ it is puzzling that a factor of $1/\sqrt{m}$ does not appear in your final formula. Would that be a typographical error? $\endgroup$ – whuber Jul 7 at 12:47
  • $\begingroup$ Dear @whuber, yes, of course, it was a typo. Thank you. $\endgroup$ – ABK Jul 7 at 12:49
  • $\begingroup$ Okay. Could you now explain what this limit means? After all, $D_{n,m}$ is not a sequence of numbers: it depends on (a) the random sample of size $m$ and (b) a presumed sequence of independent random samples of increasing sizes. It makes little sense to keep the first random sample fixed, at least if you want this limit to tell you anything useful about the test. Thus, do you intend to take expectations of $D_{n,m},$ or is your limit some kind of limit of sequences of random variables? And which sequence, exactly? $\endgroup$ – whuber Jul 7 at 13:27
  • 1
    $\begingroup$ It's unclear what you mean by "both independent sequences" because you refer to a limit where only $n$ varies. It sounds like you want to consider what happens to the KS test result when one of the samples is large. In that case--intuitively, but easily proven mathematically--the power will be limited by the size of the smaller sample. This is a general result for any statistical comparison of two independent samples. It's unclear what you mean by "asymptotically powerful" in such a situation. $\endgroup$ – whuber Jul 7 at 16:12
  • 1
    $\begingroup$ Yes, that's a definition--but it isn't apparent how you are attempting to apply that concept and, by keeping the size of one dataset fixed, it doesn't look like you are applying it appropriately. $\endgroup$ – whuber Jul 7 at 19:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.