Whitney Mann U test (i.e., Wilcoxon Rank Sum) and discrete ordinal data

Question

I want to use the Whitney Mann U test (i.e. Wilcoxon Rank Sum) test to test for differences in an ordinal outcome variable (say a 5 point Likert item) among two randomly selected samples. I've read a few articles online and I'm not sure what challenges are introduced when the data are in fact ordinal. Hopefully you all can help me understand the implications of working with ordinal data.

A page hosted by the stats department at Purdue and another hosted by the Institute for Digital Research and Education at UCLA claim we can use the Mann-Whitney U test when:

1. Comparing two samples.
2. The two groups of data are independent.
3. The type of variable could be continuous or ordinal.
4. The data might not be normally distributed.

However, a STA3024 file hosted by the stats department at UFL, a STAT464 page hosted by the stats department at PennState, a STAT105 file hosted by the stats department at UCLA, and an article hosted by the stats department at Iowa -- Ames claim this test is only appropriate for observations drawn from continuous distributions. How does the actual discrete reality of an ordinal sample affect the validity (and hypotheses) of this test (if at all)?

While the UFL file claimed continuity was important, it goes on to show an example of the Wilcoxon Rank Sum test (i.e., the Whitney-Mann U test) using responses that fall on a 5 point Likert scale. In fact, it argues that using this test with ordinal data is reasonable since the WMU test "uses only the order of the responses, not their actual values." What is happening with these data and this test to make this valid?

Answer 1

If you want to compare Likert item data from two groups with methods that I believe no one will object to, you have a couple of options.

One is ordinal regression, which is very flexible for experimental design, and is relatively easy in some software packages.

Another is the Cochran-Armitage test. The traditional form can compare only two groups, but some implementations can handle more than two groups.

You will find different opinions on using traditional nonparametric tests like Wilcoxon–Mann–Whitney (WMW) on Likert item data.

From what I can gather, the common objections to using WMW with Likert item data are a) the test has an assumption of a continuous dependent variable, and b) the test may not behave well when there are many ties in the data (as would be case for Likert data.)

From what I can gather, the common defenses for using WMW with Likert item data is that a) the test is fine handling ordinal data, and b) the test accounts for ties, at least in modern implementations. I have also heard the argument that Likert item data represents a latent continuous variable, and so doesn't violate the continuity assumption.

I'm not a statistician, so I won't attempt to evaluate these arguments.

In my experience, the traditional nonparametric tests are generally well-behaved with Likert item data. At the bottom of the page, there are simulations here comparing WMW and Kruskal–Wallis to ordinal regression.

I also think that the hypothesis that WMW tests, that of stochastic equality, makes sense in many situations with Likert item data.

As a final note, I think the advice of @DavidSmith --- using a chi-square test of association for Likert item data --- is usually not a good approach. The problem with this approach is that it discards the information about the ordinal nature of the data, and tests a hypothesis, I think, that is not generally what the analyst is interested in.

Answer 2

Neither web site is entirely wrong but neither gives you the full story, either.

I also think you are confusing the names of somewhat different tests. The Mann-Whitney and the Wilcoxon test are essentially the same and they are used to compare two distinct samples. The signed rank test, an extension of the the Wilcoxon test.

I am writing this under the impression that you have two samples, each with the same variable measured on a scale or 1, 2, 3, 4, 5. The tests you mention have no bearing.

The basic test you need is an old-fashioned chi-square test for independence. This is a general-purpose test, however, and does not take the ordering of the five categories into account. There are several proposed procedures that take the ordering into account. It isn't clear which method to use or if the ordering is that important to account for.