Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 3 down vote favorite

Suppose I use K-S to figure out if the CDF of $X$ is greater than the CDF of $Y$. I get the statistic $D^+ = \max_u\{C_x(u) - C_y(u)\}$ where $C_x$ is the ECDF for $X$ and similarly for $C_y$.

There are various tables which translate this into a p-value. Intuitively, if I'm testing if $C_x < C_y$ it seems like the null hypothesis should be $C_x \geq C_y$. (If it were $C_x = C_y$ then I don't understand how it would differ from a two-sided test.)

Yet for my data set I find low p-values for both the proposition that $C_x < C_y$ and $C_x > C_y$, leading me to believe that the null hypothesis is not the complement of the alternative.

Am I completely misunderstanding something?

share|improve this question
1  
When you write ""$C_x \ge C_y$," you probably mean that $C_x(u) \ge C_y(u)$ for all $u$. Its negation (there exists a $u$ for which $C_x(u) \lt C_y(u)$) is not the same as $C_x(u) \lt C_u(u)$ (i.e., $C_x(u) \lt C_y(u)$ for all $u$). – whuber♦ Nov 13 '12 at 5:59
1  
@whuber: good point, I was using $<$ in a vague way. Does $H_0 = \exists u: C_x(u) < C_y(u)$ for K-S? That seems like a much weaker hypothesis than what I would've expected. – Xodarap Nov 13 '12 at 13:53

1 Answer

up vote 2 down vote

I think most of the tables providing p-values for the K-S statistic are based on a two-sided test. The null hypothesis assumed by the values in the table is that the two samples are drawn from the same distribution (ie, that $C_x=C_y$). So really the table is only concerned with the absolute value of the difference between $C_x$ and $C_y$ and not the sign. That's why it does not matter if your result shows $C_x<<C_y$ or $C_x>>C_y$. Both are considered strong evidence against the null hypothesis, with a small p-value.

Let's say your null hypothesis is $C_x \leq C_y$ and your desired criticality level is $\alpha$. You could adapt the values in the table by finding the critical value of $D+$ corresponding to $2\alpha$ and using that instead. This works because the table is splitting up the probability densities into the two tails, so by doubling the specified total tail density, you are "tricking" it into allocating $\alpha$ into the upper tail, which is what you want in the one-sided test.

share|improve this answer
1  
I suspect this approach cannot work in general, because $D^+$ and $D^-$ are correlated statistics and also $D=\max(D^+,D^-)$ will not have the same distribution as either one of them. If you can provide the theory to show otherwise, then please by all means do so. In the meantime, I have conducted some simulations--and they indicate your recommendation works very well (+1). – whuber♦ Nov 13 '12 at 21:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.