1
$\begingroup$

I am trying to perform the ks.boot analysis to compare two numerical dstributions for 2 different phenomena. Those 2 numerical lists are at http://pastebin.com/s6ATRLDC and http://pastebin.com/hEc7Mqhp

When I perform the ks.boot analyses, I get the following info on-screen:

> in1=read.table("Aco195_INTERGENE-BEST-sister.UNalnPseudoCDS_Len", header=TRUE)
> in2=read.table("Aco195_PSEUDOGENE-BEST-sister.UNalnPseudoCDS_Len", header=TRUE)
> NDM=in1[,1]

> DM=in2[,1]

> summary(NDM)   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   33.0   132.0   189.0   265.1   273.0  4332.0 
> summary(DM)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
     33     159     243     358     408    4353 

> ks1<-ks.boot(NDM,DM,alternative=c("g"))
> summary(ks1)
Bootstrap p-value:     < 2.22e-16 
Naive p-value:         1.4532e-270 
Full Sample Statistic: 0.20158 

> ks2<-ks.boot(NDM,DM,alternative=c("l"))
> summary(ks2)
Bootstrap p-value:     < 2.22e-16 
Naive p-value:         2.2676e-05 
Full Sample Statistic: 0.026446 

> ks.boot(NDM,DM,alternative=c("l"))
$ks.boot.pvalue
[1] 0

>$ks

    Two-sample Kolmogorov-Smirnov test

>data:  Tr and Co
D^- = 0.0264, p-value = 2.268e-05
alternative hypothesis: the CDF of x lies below that of y


>$nboots
[1] 1000

>attr(,"class")
[1] "ks.boot"


> ks.boot(NDM,DM,alternative=c("g"))
>$ks.boot.pvalue
[1] 0

>$ks

    Two-sample Kolmogorov-Smirnov test

>data:  Tr and Co
D^+ = 0.2016, p-value < 2.2e-16
alternative hypothesis: the CDF of x lies above that of y

>$nboots
[1] 1000

>attr(,"class")
[1] "ks.boot"

I am confused because it seems both p-values are low enough to allow accepting the alternative hypothesis and reject the NULL. However, that is absurd because one numerical distribution cannnot be both higher and lower than the other! So obviously I am interpreting this wrong. I dont have much stats background, so hopefully someone can spell it out.

I have not had much luck trying to see how content posted at Goodness of fit test for a mixture in R or ks.test and ks.boot - exact p-values and ties can help clarify my confusion. In my mind, its as though both alternative hypothesis ("g" and "l") are acceptable, and that one is relatively even less likely to occur by chance than the other, so calculating the ratio of these 2 p-values would give something that may be meaningful? Or rather I should choose the alternative hypothesis that yields the lowest p-value? Yes? Or may be I am totally off. Help, please!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

This is because of the absolute value that is used as test statistic:

$$D_n= \sup_x |F_n(x)-F(x)|$$

Source

To get a "correct" result you could use a two sided test, which will simply tell you whether your distributions are different or not.

Here is some discussion about it.

They also provide another useful link about this topic

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks Verena, for your rely & links in it. I have 48 pairwise comparisons, each similar to the 1 example pairwise data I posted. Should I visually examine the distributions for each pair, like Glen_B does in his response below? And use these graphs to observe whether "switchback" occurs in a certain pairwise comparison. And if yes, then rather than use KS, I should use the Wilcoxon Rank Sum Test? Even if switchback does not occur for a certain pairwise comparison, then can/should I still use the Wilcoxon Rank Sum Test? Seems like I should use just 1 test across all pairwise comparisons, yes? $\endgroup$
    – AksR
    Commented Jul 14, 2015 at 17:04
2
$\begingroup$

However, that is absurd because one numerical distribution cannnot be both higher and lower than the other!

Yes, actually it can:

enter image description here

With a pair of distributions like that you could reject in one direction near -2 and reject in the other direction near 2.

And that looks something like what's going on with your data:

enter image description here

While the two gaps are quite different in size, the smaller gap is still highly significant. (The "100 times" is only a rough ballpark figure; it's probably a good deal more than 100 times as large as the 2.5% point)

Improve this answer
$\endgroup$
4
  • $\begingroup$ This is a very interesting observation, with possible biological meaning, Thanks! I am now going to check whether I see the same switchover in the remaining 47 pairwise comparisons. If yes, can I split such datasets at the switchover point, and then perform the ks.boot test separately on them ? In theory, that would allow "lesser" for one side and "greater" for the other side to be the only acceptable alternate hypothesis over Null (= both distributions are equal). Correct? If it makes sense, please suggest R syntax to split data for performing separate ks.boot tests on them. Thanks Glen_b! $\endgroup$
    – AksR
    Commented Jul 14, 2015 at 18:54
  • $\begingroup$ I'd suggest that you ask about whether the splitting is acceptable as a new question (and what else one might do if it isn't); it's not a one- or two-line answer (and it might be good to get other views). $\endgroup$
    – Glen_b
    Commented Jul 15, 2015 at 0:28
  • $\begingroup$ Very well, thank you Glen_b. I will do as you suggested and start a new thread. Thanks for your help. $\endgroup$
    – AksR
    Commented Jul 15, 2015 at 2:26
  • $\begingroup$ You can always link back to this question in your new one for additional context. $\endgroup$
    – Glen_b
    Commented Jul 15, 2015 at 2:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.