5
$\begingroup$

In a previous question, I asked about comparing the power of a t test to a Mann Whitney test under different situations. One of the answers pointed out that the worst that the Mann-Whitney can ever perform relative to the t-test is that it would require 1/0.864x as much data to give the same power as the t test so long as the data sets being compared were from the same distribution.

I guess I am confused about how power can be compared between these two tests given that they test different null hypotheses. For example, if I estimate the power for a t-test at 0.9 using effect size equal to a 20% difference between means, then that makes sense for a t test. But a Mann Whitney test does not test for differences between means. If the distributions were the same it would test for differences between medians. Am I right in thinking that the Mann Whitney test would require 1/0.864x as much data to detect a 20% difference in medians with a power of 0.9?

$\endgroup$
2
$\begingroup$

1) It's important to distinguish between an estimator and the population quantity it is being used to estimate.

For example, for a symmetric distribution whose mean exists, I could use any number of sample estimators of the population mean - such as the sample median, for example. [And, for example, if the distribution is Laplace, the sample median is actually a substantially better estimate of the population mean than the sample mean is.]

2) If you assume the same distribution-shape (i.e. identical apart from a possible location shift) for the two populations in your test, then both the Mann-Whitney and the t-test test for that location shift, which is the same shift whether you think of it as a shift in means, medians, 95th percentiles or whatever.

In that situation, the Mann-Whitney is as much a test of difference in populations means as the t-test is, it just uses a different estimator for the quantity. Indeed, it's quite possible to produce both an estimate of (and interval for) that shift under either test.

The Mann-Whitney estimate of the shift is the median of pairwise cross-sample differences (see the second paragraph here) -- but similar to the example in (1), that's a perfectly valid way to estimate the shift in population means.

For more discussion of some of these issues, and how a Mann-Whitney and a t-test test the same null and alternative hypothesis once you assume that the alternative is location-shift, see this answer

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.