Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I would like to compare means/medians of two samples that may be very skewed in size, e.g. 25 points and 1 or 2 points and test them for similarity, rather than difference. I realize that the power of such a test is likely to be low, however I would like to calculate it nevertheless. I have been reading about equivalence tests but so far haven't seen much on the non-parametric side. Because of the small sample size, if there are ideas for permutation/simulation/bootstrap tests for similarity, I'd like to hear those too. Thanks much.

share|improve this question

Non-parametric tests such as the ones developed by Wilcoxon would be appropriate for such a task.

To analyze goodness of fit between two large sets you can consider: non-parametric tests such as Anderson-Darling or Kolmogorov-Smirnov. Anderson-Darling is pretty good for running samples of different sizes (if your samples get larger than just 1 or 2), in particular. HTH.

share|improve this answer
Welcome to the site, @JackRyan. There's no need to ask for people to comment on your posts, people will do that anyway if they feel it's merited whether you mention it or not. Since you're new here, you may want to read our FAQ. – gung Feb 12 '13 at 4:02
Jack, please let me add my welcome to @gung's. Would you mind explaining how the AD or KS tests could be used to "compare means/medians"? I understand them to be tests of Goodness of Fit (to particular distributions), which amounts to a lot more than just comparing measures of location. – whuber Feb 12 '13 at 4:30
Thanks for the guidance. I skipped over what was being asked. . . – Jack Ryan Feb 12 '13 at 13:12
up vote 1 down vote accepted

I found lots of relevant information in a paper and subsequent book by Stefan Wellek:

A New Approach to Equivalence Assessment in Standard Comparative Bioavailability Trials by Means of the Mann-Whitney Statistic

Testing Statistical Hypotheses of Equivalence and Noninferiority, Second Edition

Unfortunately, neither of these sources are free. The book is supposed to have SAS/R code associated with it but the provided link is broken. The text does provide enough info to implement it though.

share|improve this answer

For the (still) interested R-user:

Assume you want to test at the 5% level the working hypothesis of "true shift D within -d and d", where d is the equivalence margin. Use the R-function "wilcox.test" to obtain a 90% c.i. for D (=median of the difference between a sample from group 1 and a sample from group 2). If this c.i. is contained entirely in [-d,d], then you could be 95% certain that your working hypothesis is true.

Remark: If you can show (somehow) distributional equivalence using e.g. 2-sample-KS-test, equivalence in any sort of location parameter follows.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.