I would like to compare means/medians of two samples that may be very skewed in size, e.g. 25 points and 1 or 2 points and test them for similarity, rather than difference. I realize that the power of such a test is likely to be low, however I would like to calculate it nevertheless. I have been reading about equivalence tests but so far haven't seen much on the non-parametric side. Because of the small sample size, if there are ideas for permutation/simulation/bootstrap tests for similarity, I'd like to hear those too. Thanks much.
$\begingroup$
$\endgroup$
asked Feb 11, 2013 at 22:40
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
For the (still) interested R-user:
Assume you want to test at the 5% level the working hypothesis of "true shift D within -d and d", where d is the equivalence margin. Use the R-function "wilcox.test" to obtain a 90% c.i. for D (=median of the difference between a sample from group 1 and a sample from group 2). If this c.i. is contained entirely in [-d,d], then you could be 95% certain that your working hypothesis is true.
Remark: If you can show (somehow) distributional equivalence using e.g. 2-sample-KS-test, equivalence in any sort of location parameter follows.
$\begingroup$
$\endgroup$
3
Non-parametric tests such as the ones developed by Wilcoxon would be appropriate for such a task.
To analyze goodness of fit between two large sets you can consider: non-parametric tests such as Anderson-Darling or Kolmogorov-Smirnov. Anderson-Darling is pretty good for running samples of different sizes (if your samples get larger than just 1 or 2), in particular. HTH.
-
1$\begingroup$ Welcome to the site, @JackRyan. There's no need to ask for people to comment on your posts, people will do that anyway if they feel it's merited whether you mention it or not. Since you're new here, you may want to read our FAQ. $\endgroup$ Commented Feb 12, 2013 at 4:02
-
1$\begingroup$ Jack, please let me add my welcome to @gung's. Would you mind explaining how the AD or KS tests could be used to "compare means/medians"? I understand them to be tests of Goodness of Fit (to particular distributions), which amounts to a lot more than just comparing measures of location. $\endgroup$– whuber ♦Commented Feb 12, 2013 at 4:30
-
$\begingroup$ Thanks for the guidance. I skipped over what was being asked. . . $\endgroup$ Commented Feb 12, 2013 at 13:12
$\begingroup$
$\endgroup$
I found lots of relevant information in a paper and subsequent book by Stefan Wellek:
Testing Statistical Hypotheses of Equivalence and Noninferiority, Second Edition
Unfortunately, neither of these sources are free. The book is supposed to have SAS/R code associated with it but the provided link is broken. The text does provide enough info to implement it though.
answered Feb 20, 2013 at 16:47