# non-parametric two-sample equivalence tests with unequal sample sizes

I would like to compare means/medians of two samples that may be very skewed in size, e.g. 25 points and 1 or 2 points and test them for similarity, rather than difference. I realize that the power of such a test is likely to be low, however I would like to calculate it nevertheless. I have been reading about equivalence tests but so far haven't seen much on the non-parametric side. Because of the small sample size, if there are ideas for permutation/simulation/bootstrap tests for similarity, I'd like to hear those too. Thanks much.

For the (still) interested R-user:

Assume you want to test at the 5% level the working hypothesis of "true shift D within -d and d", where d is the equivalence margin. Use the R-function "wilcox.test" to obtain a 90% c.i. for D (=median of the difference between a sample from group 1 and a sample from group 2). If this c.i. is contained entirely in [-d,d], then you could be 95% certain that your working hypothesis is true.

Remark: If you can show (somehow) distributional equivalence using e.g. 2-sample-KS-test, equivalence in any sort of location parameter follows.

Non-parametric tests such as the ones developed by Wilcoxon would be appropriate for such a task.

To analyze goodness of fit between two large sets you can consider: non-parametric tests such as Anderson-Darling or Kolmogorov-Smirnov. Anderson-Darling is pretty good for running samples of different sizes (if your samples get larger than just 1 or 2), in particular. HTH.

• Welcome to the site, @JackRyan. There's no need to ask for people to comment on your posts, people will do that anyway if they feel it's merited whether you mention it or not. Since you're new here, you may want to read our FAQ. Feb 12 '13 at 4:02
• Jack, please let me add my welcome to @gung's. Would you mind explaining how the AD or KS tests could be used to "compare means/medians"? I understand them to be tests of Goodness of Fit (to particular distributions), which amounts to a lot more than just comparing measures of location.
– whuber
Feb 12 '13 at 4:30
• Thanks for the guidance. I skipped over what was being asked. . . Feb 12 '13 at 13:12

I found lots of relevant information in a paper and subsequent book by Stefan Wellek:

A New Approach to Equivalence Assessment in Standard Comparative Bioavailability Trials by Means of the Mann-Whitney Statistic

Testing Statistical Hypotheses of Equivalence and Noninferiority, Second Edition

Unfortunately, neither of these sources are free. The book is supposed to have SAS/R code associated with it but the provided link is broken. The text does provide enough info to implement it though.