Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

I've got a bunch of data from two samples (control and treated), each containing several thousand values which are to undergo significance testing in R. Theoretically, the values should be continuous, but due to rounding done by measurement software they aren't and they have got ties. The distributions are unknown and the shapes of control and treated distributions might be different, so I'd like to use a non-parametric test to compare if the difference across the samples is significant for 10 different factors.

I thought of using the Kolmogorov-Smirnov test, but it's not really suitable for ties. I recently stumbled upon a new R library called Matching that executes a bootstrap version of K-S test and tolerates ties. Now is this really a good idea or should I use another test instead? And do I need to adjust the p-value?

share|improve this question
The linked paper deals with propensity score matching. It may be that the bootstrap test has more generality but I am not sure. – Michael Chernick Sep 3 '12 at 14:17

1 Answer

up vote 6 down vote accepted

Instead of using the KS test you could simply use a permutation or resampling procedure as implemented in the oneway_test function of the coin package. Have a look at the accepted answer to this question.

Update: My package afex contains the function compare.2.vectors implementing a permutation and other tests for two vectors. You can get it from CRAN:

install.packages("afex")

For two vectors x and y it (currently) returns something like:

> compare.2.vectors(x,y)
$parametric
   test test.statistic test.value test.df       p
1     t              t     -1.861   18.00 0.07919
2 Welch              t     -1.861   17.78 0.07939

$nonparametric
             test test.statistic test.value test.df       p
1 stats::Wilcoxon              W     25.500      NA 0.06933
2     permutation              Z     -1.751      NA 0.08154
3  coin::Wilcoxon              Z     -1.854      NA 0.06487
4          median              Z      1.744      NA 0.17867

Any comments regarding this function are highly welcomed.

share|improve this answer
2  
(+1) A description of this and other tests can be found in this blog – user10525 Sep 3 '12 at 15:25
@Henrik Thanks for the suggestion and for pointing to the other question. That's really helpful! – AnjaM Sep 4 '12 at 11:41
@AnjaM You are welcome. You might also want to check my update. – Henrik Sep 4 '12 at 11:47

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.