7
$\begingroup$

Suppose I use K-S to figure out if the CDF of $X$ is greater than the CDF of $Y$. I get the statistic $D^+ = \max_u\{C_x(u) - C_y(u)\}$ where $C_x$ is the ECDF for $X$ and similarly for $C_y$.

There are various tables which translate this into a p-value. Intuitively, if I'm testing if $C_x < C_y$ it seems like the null hypothesis should be $C_x \geq C_y$. (If it were $C_x = C_y$ then I don't understand how it would differ from a two-sided test.)

Yet for my data set I find low p-values for both the proposition that $C_x < C_y$ and $C_x > C_y$, leading me to believe that the null hypothesis is not the complement of the alternative.

Am I completely misunderstanding something?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ When you write ""$C_x \ge C_y$," you probably mean that $C_x(u) \ge C_y(u)$ for all $u$. Its negation (there exists a $u$ for which $C_x(u) \lt C_y(u)$) is not the same as $C_x(u) \lt C_u(u)$ (i.e., $C_x(u) \lt C_y(u)$ for all $u$). $\endgroup$
    – whuber ♦
    Commented Nov 13, 2012 at 5:59
  • 1
    $\begingroup$ @whuber: good point, I was using $<$ in a vague way. Does $H_0 = \exists u: C_x(u) < C_y(u)$ for K-S? That seems like a much weaker hypothesis than what I would've expected. $\endgroup$
    – Xodarap
    Commented Nov 13, 2012 at 13:53

1 Answer 1

Reset to default
2
$\begingroup$

I think most of the tables providing p-values for the K-S statistic are based on a two-sided test. The null hypothesis assumed by the values in the table is that the two samples are drawn from the same distribution (ie, that $C_x=C_y$). So really the table is only concerned with the absolute value of the difference between $C_x$ and $C_y$ and not the sign. That's why it does not matter if your result shows $C_x<<C_y$ or $C_x>>C_y$. Both are considered strong evidence against the null hypothesis, with a small p-value.

Let's say your null hypothesis is $C_x \leq C_y$ and your desired criticality level is $\alpha$. You could adapt the values in the table by finding the critical value of $D+$ corresponding to $2\alpha$ and using that instead. This works because the table is splitting up the probability densities into the two tails, so by doubling the specified total tail density, you are "tricking" it into allocating $\alpha$ into the upper tail, which is what you want in the one-sided test.

Improve this answer
$\endgroup$
4
  • 1
    $\begingroup$ I suspect this approach cannot work in general, because $D^+$ and $D^-$ are correlated statistics and also $D=\max(D^+,D^-)$ will not have the same distribution as either one of them. If you can provide the theory to show otherwise, then please by all means do so. In the meantime, I have conducted some simulations--and they indicate your recommendation works very well (+1). $\endgroup$
    – whuber ♦
    Commented Nov 13, 2012 at 21:56
  • $\begingroup$ @whuber Do you know of a CDF for $D^{+}$ or $D^{-}$? I am having a the dickens of a time tracking one down. $\endgroup$
    – Alexis
    Commented Jul 12, 2014 at 17:26
  • 1
    $\begingroup$ @Alexis Using the probability integral transform, Donald Knuth derives their (common) distribution on p. 57 and exercise 17 of TAoCP Volume 2. I quote: $$\Pr\left(D_n^{+}\le \frac{t}{\sqrt{n}}\right)=\frac{t}{n^n}\sum_{0\le k\le t}\binom{n}{k}(k-t)^k(t+n-k)^{n-k-1}.$$ It is asymptotically Normal for large $n$. $\endgroup$
    – whuber ♦
    Commented Jul 14, 2014 at 15:44
  • $\begingroup$ Thank you @whuber I did find this one sample CDF, and have expanded my inquiry for a two-sample CDF to a question: stats.stackexchange.com/questions/107805/… $\endgroup$
    – Alexis
    Commented Jul 14, 2014 at 16:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.