non-parametric two-sample equivalence tests with unequal sample sizes I would like to compare means/medians of two samples that may be very skewed in size, e.g. 25 points and 1 or 2 points and test them for similarity, rather than difference.  I realize that the power of such a test is likely to be low, however I would like to calculate it nevertheless.  I have been reading about equivalence tests but so far haven't seen much on the non-parametric side.  Because of the small sample size, if there are ideas for permutation/simulation/bootstrap tests for similarity, I'd like to hear those too. Thanks much.  
 A: For the (still) interested R-user:
Assume you want to test at the 5% level the working hypothesis of "true shift D within -d and d", where d is the equivalence margin. Use the R-function "wilcox.test" to obtain a 90% c.i. for D (=median of the difference between a sample from group 1 and a sample from group 2). If this c.i. is contained entirely in [-d,d], then you could be 95% certain that your working hypothesis is true.
Remark: If you can show (somehow) distributional equivalence using e.g. 2-sample-KS-test, equivalence in any sort of location parameter follows.
A: Non-parametric tests such as the ones developed by Wilcoxon would be appropriate for such a task.  
To analyze goodness of fit between two large sets you can consider: non-parametric tests such as Anderson-Darling or Kolmogorov-Smirnov.  Anderson-Darling is pretty good for running samples of different sizes (if your samples get larger than just 1 or 2), in particular.  HTH.
A: I found lots of relevant information in a paper and subsequent book by Stefan Wellek:
A New Approach  to Equivalence Assessment in Standard Comparative Bioavailability Trials by Means of the Mann-Whitney Statistic
Testing Statistical Hypotheses of Equivalence and Noninferiority, Second Edition
Unfortunately, neither of these sources are free.  The book is supposed to have SAS/R code associated with it but the provided link is broken. The text does provide enough info to implement it though.  
