# "Strength" of hypothesis tests

I was referred to this site from mathoverflow. After some brief searching, I haven't seen another thread on my particular question, but if it's already been answered elsewhere then I hope you'll forgive the spam.

Has anyone ever "ranked" the various commonly-used parametric and non-parametric hypothesis tests in terms of the "strength" of the conclusions that one can ascertain from them? That is to say, if I have two populations that are wildly different from each other (and about which I have absolutely no assumptions whatsoever on the prior distributions), and I'd like to make the strongest possible case that they are different, what would be the right hypothesis test to use?

An example would be if I have one population of samples that all lie between $0$ and $1$ and another population of samples that all lie between $10$ and $11$. Running a $t$-test would likely show that my sample means are different, but I would think that I could make stronger conclusions than that (based on the fact that largest element in one set is smaller than the smallest set in the other, for example).

(as an aside, in the particular case I'm dealing with, one population has lots of samples and the other is quite small, and both have rather small standard deviations)

• I find your example a little confusing, John. It refers to a situation with extremely strong distributional assumptions, but it immediately follows a remark about having "absolutely no assumptions whatsoever on the prior distributions," which sounds like exactly the opposite situation. Which is the situation you really have in mind? Also, it is not clear what you mean by "strength." Initially it would seem to refer to force of evidence, but in the example it instead seems to mean the degree of detail (such as going beyond just comparing means). Could you make that idea more precise?
– whuber
Commented Dec 30, 2011 at 22:38
• Thanks, whuber -- I should have made that more clear. I edited the example to better reflect the question I asked. I'll give careful thought to what exactly I meant by "strength" and edit the question accordingly in the (very) near future. Commented Dec 30, 2011 at 23:37