If you analyze the same data with a t test and the nonparametric Mann-Whitney test, which do you expect to have the lower P value?

  • $\begingroup$ Someone asked me this, so I am posting both the question and my answer. $\endgroup$ – Harvey Motulsky Aug 31 '12 at 15:19

It depends.

If you assume that the data are sampled from Gaussian distributions, then the t test has a bit more power (depending on sample size) so will -- on average -- have a lower P value. But only on average. For any particular set of data, the t test may give a higher or a lower P value.

If you don't assume the data are sampled from Gaussian distributions, then the Mann-Whitney test may have more power (depending on how far the distribution is from Gaussian). If so, you'd expect the Mann-Whitney test to have the lower P value on average, but the results are not predictable for any particular set of data.

What does "on average" mean? Perform both tests on many sets of (simulated) data. Compute the average P value from the t test, and also the average P value from the Mann-Whitney test. Now compare the two averages.

  • $\begingroup$ Is there Monte Carlo research or an analytic solution on which you based this answer? $\endgroup$ – Joel W. Aug 31 '12 at 15:46
  • 3
    $\begingroup$ @JoelW: an analytic approach to the relative power of the Wilcoxon-Mann-Whitney and t-test is discussed in this question. As Harvey says, for data from a normal distribution the t-test has more power in large samples - it's relative power versus the U test is $\pi/3$ - but for non-normal data, the U test can have rather more power (3 times on exponential, 1.5 on double exponential, infinitely so on Cauchy). For uniformly distributed data, they're equally powerful. $\endgroup$ – Silverfish Jan 1 '15 at 22:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.