3
$\begingroup$

I am a high school Psychology student. I just performed a study where two separate groups of sizes 26 and 28 took the same test under different conditions. For the data analysis my professor has instructed me to use the Mann-Whitney test, which is unfortunate because both have sample sizes greater than 20. I saw something on the Wikipedia article about how the sample distributions are approximately normal at that point, but I have no idea how to use the z variable it gives me because I hardly know anything about statistics. Can somebody give a noob a simple process for performing the Mann-Whitney test correcting for the sample size?

$\endgroup$
  • $\begingroup$ There is no requirement that sample size for Mann-Whitney be smaller than 20. $\endgroup$ – Sal Mangiafico Jul 17 '18 at 13:50
  • $\begingroup$ Yeah, seemed arbitrary to me, especially when we were instructed to throw out data to reach that sample size! $\endgroup$ – jkmartindale Jul 17 '18 at 14:39
  • $\begingroup$ Oh my. If that was a public school, I would ask for my taxes back. $\endgroup$ – Sal Mangiafico Jul 17 '18 at 14:43
  • $\begingroup$ Glad to hear it worked out. $\endgroup$ – Sal Mangiafico Jul 17 '18 at 15:06
2
$\begingroup$

The unpaired t-test is efectively "how to use the z variable" provided by the normal approximation, as you asked. In many circumstances the t-test (based on underlying normal distributions) may be as good as or better than the Mann-Whitney test, and as @AdamO notes the two tests tend to give similar results in practice.

But you have been asked to use the Mann-Whitney test, and there's no reason not to give it a try. There's no theoretical problem in using the Mann-Whitney test with samples of this (or any) size, and there's no need to "correct" for sample size. For example, the accepted answer on this page argues "you can't shoot yourself in the foot by using the Mann-Whitney test instead of the t-test, but the converse is not true." The Mann-Whitney test is based on comparisons of ranks among cases, rather than the actual values. This means it doesn't make strong assumptions about the underlying distributions, but it does raise some interesting issues having to do with ties, as discussed on this page. Modern statistical packages have ways of taking ties into account.

As you are just starting to learn how to apply statistics to real problems, you might find it informative to use both the Mann-Whitney test and an unpaired t-test on your data. Also, you should know that most analysts wouldn't consider this a "large" data set; that usually means more like thousands of cases (some might say millions) and up. But yes, this is the size of samples for which normal approximations are often good enough.

$\endgroup$
1
$\begingroup$

There are potentially two problems you are assessing: 1. The T-distribution's approximation to the normal curve in large N and 2. the use/misuse of the Mann Whitney test when distributional assumptions are not met (based on qualitative inspection).

For 1:

The Mann-Whitney test is just a T-test that replaces the actual observed values with their relative rank in a pooled sample. So, for instance, the smallest (or largest, it doesn't matter) value in the combined 54 observation sample is assigned value 1,the next in order the value 2, and so on. Then a T-test is applied to compare a difference in ranks.

We use a T-test instead of a Z-test when the variance of a sample is to be estimated from the same data whence we estimate the mean. This is to account for the bias that comes from "using the data twice". This bias correction is expressed as a number of degrees of freedom, for instance, a DF of 5 will give you a T-distribution that is very "stout" (large SD relative to a normal curve). As the DF of a T-distribution become numerous (say, 50 or more), the distribution becomes very regularly normal.

Arguably, when the T-statistic is appropriate in small samples, it should never be relaxed based on any condition of the sample size. If the degrees of freedom are large enough to give sampling distributions to test statistics which are approximately normal, we can be confident the T-distribution will reflect that as a consequence of its numerous degrees of freedom.

For 2:

Often in science, the T-test is taught that it is a test of mean differences in normally distributed data. It is true that this test will be exact when this is the case. However, if the data are non-normal, the t-test is "robust" to departures from normality. So even non-normal data, like proportions, test scores, ranks, etc. can be tested for mean differences and the t-test does "pretty good" (this is actually a mathematically rigorous definition).

The Mann-Whitney test was introduced as one of many statistical alternative tests based on the "ranks" of the data, as I described earlier. Therefore, many people touted such tests as being robust to outliers (which are NOT rigorously defined and scientifically dubious). This was further taken to mean that these tests can be applied when distributional assumptions about normality are violated.

It turns out that the Mann Whitney and the T-test usually give very similar inference. In fact, the t-test is more efficient (meaning that, if one were to sample more and more data, the t-test gets you the right answer quicker). The Mann-Whitney test, in fact, does rely on some assumptions (such as symmetric distributions) in order to make sure the inference is correct (about differences in medians). For this reason, I usually prefer running plain vanilla t tests for nearly all inference.

Mann-Whitney (for reasons unknown to me) became popular for small sample inference. I suspect it's because it's difficult to inspect histograms for approximate normality in small samples. However, due to the efficiency consideration I mentioned before, I think this only serves to give one a more conservative test (which is equivalent to throwing data away).

$\endgroup$
  • $\begingroup$ "The [MW] test... [relies] on some assumptions (such as symmetric distributions) in order to make sure the inference is correct (about differences in medians)." The about differences in medians here is critical to the claim. The usual null hypothesis of the Mann-Whitney test --- that of stochastic equality --- doesn't make assumptions about the shape of the distributions. It is only if we want to test a different hypothesis, about the median, that we need to make these assumptions. I don't know why people want to constrain the perfectly useful and interesting MW test in this manner. $\endgroup$ – Sal Mangiafico Jul 17 '18 at 14:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.