Mann-Whitney or t-test to compare age and expenditure after clustering

Question

I have carried out cluster analysis and now want to compare means between variables in different clusters. The variables in question are age and expenditure in millions of dollars.

The age variable does not follow a normal distribution: as a result, I was considering a Mann-Whitney test. The expenditure in millions of dollars fails the assumption of equality of variances.

Having stated this, although all tests seem to suggest that age does not follow normal distribution, I am not quite sure about the extent of this.

Histogram of age in cluster 1

Histogram of Age in Cluster 2

Box plot of Age in Cluster 1

Box Plot of Age in Cluster 2

It has been suggested to use the Mann-Whitney test in this case, given that the assumption of normality is "not met".

1. Does Mann Whitney work fine with continuous data? This link seems to suggest it does. Would SPSS automatically convert these into ranks?
2. Zimmerman argues that t-test should work fine because it is scarcely affected by non-normality of the population!
3. Sheskin (2007) suggests using a t-test anyway but using a more conservative approach (e.g critical values of t(0.01) instead of t(0.05).

How can I resolve this problem?

Answer 1

There are many questions on this already, just have a look using the search function. Some details of your questions however seem to warrant some specific remarks:

• Mann-Whitney U test works fine with continuous data, I would even say it works best with them because you would avoid ties.
• The t-test has indeed been found robust to some violations of its assumptions but not to all of them, especially if they happen concurrently. Larger sample sizes help to relax these constraints. You can find many information on this elsewhere on this site.
• Point 3 is surprising. For one, the whole point of a test is to offer some guarantees regarding the error level, provided the assumptions are met. If you can't achieve that, just picking an arbitrary “conservative” level just muddles the situation further. Better give up the test entirely. Furthermore, one common problem with the t-test and non-normal data is lack of power. A lower threshold just makes this problem worse. All this would seem to make the result very difficult to interpret one way or the other.
• I would generally be skeptical of tests between groups that are not defined a priori, certainly if the variables you are comparing were also used for the cluster analysis. All this sound a bit too exploratory for tests to be meaningful. You might just as well plot the data and comment what you see, understanding that you are just providing a tentative interpretation.

---

Practical recommendations in light of your comments:

• Mann-Whitney is perfectly fine but do realize it is not a test of the difference in means. It might or might not be a problem for you but the most important point is that you cannot just think of this problem as “normal data => t-test, non-normal => Mann-Whitney U”. There is a lot more going on (check the links I added as a comment to the question for more on that).
• The t-test might be fine. I already wrote that a hard-and-fast threshold would be very questionable and it's still impossible to give advice based only on the notion that the data are “non-normal”. Whether it matters or not depends on the specific ways in which they are non-normal.
• 300 observations is already quite comfortable. Do run both tests, possibly some other alternatives as well (permutation test, bootstrap test of the median or another robust estimator of location if that makes sense…). Also inspect the distribution and the residuals. You might very well find all this point to broadly similar conclusions and would not need to worry about this further.
• You said that the two variables are not the “main” predictors in the cluster analysis but are they in the analysis at all? I would still not be fully convinced of the value of the whole approach but you should at least keep them entirely separate I think.
• Don't overestimate tests. Since you are happy using an exploratory method like cluster analysis, do also plot the data and interpret that in any case.

Answer 2

What is the problem that you are trying to solve?

1. It is the job of cluster analysis to find clusters. Testing whether those clusters really exist post hoc is, in my view, somewhere between meaningless and dubious. To strip the problem down to the bare minimum, imagine one variable that is a continuum, say people's heights, and you group height values into two clusters. Now do you want to test that short people and tall people have different means? Now explain why the problem of three or more clusters and/or two or more variables makes the problem different.

2. You describe the data as being age and expenditure. If that's so, then a scatter plot showing the data will show clusters clearly if they exist and a continuum of variation if they don't. You can be flexible about scales (e.g. logging expenditure). What cluster analysis might add to this of real scientific value is an open question. Cluster analysis divides statistical people into two clusters, those who think it a central technique and those who think it oversold snake oil or worse.

3. As with #2, if you are clustering on two variables, then each cluster has two means, mean age and mean expenditure. Even if you have a good answer to #1, univariate tests don't capture the differences between clusters.

4. If #2 and #3 are wrong, and you have other variables, then the problem is not as you stated and you need to explain why.

5. If you are interested in means, then Mann-Whitney is not an alternative method of comparing means.

6. Mann-Whitney uses ranks, so it doesn't even know whether the original data are discrete or continuous. In practice, discrete data are more likely to show ties and that can have a secondary effect on the test. Whether your software adjusts for ties depends on what it is.

7. How many clusters do you have any way? Is it just two?

I can't advise on SPSS. I often advise against SPSS, but that's a prejudice.

Please give a reference for Sheskin (2007).

Answer 3

First, realize that the Mann Whitney U test and the t test test different things: The t test is a test of differences in means, the U test is a test of entire distributions. It is possible that the means could be the same and the distributions different (although this would only happen for odd distributions, as far as I can tell, for example

set.seed(12345)
x <- rnorm(1000)
y <- c(rlnorm(500), runif(500,-100,100))
wilcox.test(x,y)
t.test(x,y)

where the Wilcox test rejects at $p = 2.2 * 10^{-16}$ and the t does not reject at all.

Second, while the robustness of the t as a test of means depends on exactly how the data are non-normal, sample size, variances and so on, the fact that it is a test of means remains. If age is not normally distributed you may not want to test the means.

Answer 4

The derivation of the Mann Whitney assumes continuous data. When you have heavier-tailed than normal data, it's also typically more powerful than the t-test; if you assume only location shift alternatives, it's a test of difference in means (along with any other reasonable location measure); if that doesn't hold, it's testing something else.

That said, the t-test can tolerate moderate skewness and heavy-tailedness (though in the latter case your actual significance levels will tend to be lower than the nominal $\alpha$).

There's also the possibility of a permutation test rather than either of the choices you mention - it would allow you to test a difference in means and have it be valid when the assumptions of the t-test are not satisfied.

Another possibility if you think that some exponential family distribution might suit better (such as a gamma distribution) would be to fit a GLM with a group factor representing the groups whose means you are comparing. An identity link will even give you a direct estimate of the difference in means.