6
$\begingroup$

Consider following example. We asked 50 men and 50 women one question and they answered on 5 point likert scale (1 is completely disagree and 5 completely agree). Suppose that 50 men responded 20 times 1, 3 times 2, 4 times 3, 20 times 4 and 3 times 5 and 50 women responded 15 times 1, 15 times 2, 15 times 3, 2 times 4 and 3 times 5.

I would like to test whether mean (or median) answers in both groups are significatly different.

The problem is that I can not use parametric (t-test) neither nonparametric (Mann-Whitney) test since data are nonnormal (thus t-test is not appropriate), ordinal and thus not continuous, multimodal, with different variances and with different shapes (thus Mann-Whitney test is not appropriate). How can I test my hypothesis? Thank you for any answer.

Improve this question
$\endgroup$
5
  • $\begingroup$ I don't understand why you think the Mann-Whitney U-test can't be used here. Can you clarify that? (In fact, I think it's the appropriate choice for your situation.) $\endgroup$
    – gung - Reinstate Monica
    Commented Oct 8, 2012 at 0:21
  • 2
    $\begingroup$ For multimodal data, tests of location (mean or median) are rather inane. $\endgroup$
    – Peter Flom
    Commented Oct 8, 2012 at 0:22
  • $\begingroup$ To my best knowledge, Mann-Whitney has some assumptions (same shapes of distributions and equal variances). And my artificial example obviously does not met them. $\endgroup$
    – sitems
    Commented Oct 8, 2012 at 0:53
  • $\begingroup$ Since you have a likert scale with 5 groups, you could present tha data as a contingency table. Maybe that helps, you could also apply contingency table methods. But Peter Flom's comment still applies! $\endgroup$
    – kjetil b halvorsen ♦
    Commented Oct 8, 2012 at 0:55
  • $\begingroup$ Ok, but what in the case when both groups are unimodal but with different shapes? Contingency table methods are possible, but they work with the lowest scale, nominal. So I only wanted to know if I can test my hypothesis with at least ordinal scale. $\endgroup$
    – sitems
    Commented Oct 8, 2012 at 0:59

2 Answers 2

Reset to default
13
$\begingroup$

What you write is a compilation of many common misconceptions about these tests.

The short answer is: use the t test with Welch correction. Now, the details.

I would like to test whether mean (or median) answers in both groups are significantly different.

Means and medians are different things. What people usually do, is that they think in terms of means, not medians, so by default this is also what you should aim for. The Likert scale was invented by the guy Rensis Likert precisely with the intention to make it useful for computing means (not only medians). See the James Carifio, Rocco Perla, "Resolving the 50-year debate around using and misusing Likert scales" in commentaries in Medical Education, 2008, Blackwell Publishing Ltd.

The problem is that I can not use parametric (t-test) neither nonparametric (Mann-Whitney) test since data are nonnormal (thus t-test is not appropriate) (...), multimodal (...)

Definitely not! Both tests are robust with respect to the shape of the distribution. Only Mann-Whitney (it is usually called Wilcoxon-Mann-Whitney, or WMW test) requires both distributions to have the same shape.

(...) ordinal and thus not continuous (...)

"Ordinal variable" means "arithmetic means doesn't make sense on it". Like education measured in 3-level scale "1 - grammar school", "2 - college" and "3 - university". This does not imply it is not continuous (although usually it is the case).

Neither the WMW nor the t-test require continuous variables.

(...) with different variances (...)

When you use the t-test with Welch's correction (Welch B.L. The generalization of Student's problem when several different population variances are involved, Biometrika, 34, 28-38, 1938) than you don't need to worry about the unequal variances (and shapes). The t-test (as with any test based on means) is already very robust with respect to departures from normality (see e.g. Michael R. Chernick, Robert H. Friis "Introductory Biostatistics for The Health Sciences", Willey Interscience 2003 and many other books). This property comes from the fact, that it is based on means. By the virtue of the Central Limit Theorem, the distribution of the mean very quickly converges to normal distribution.

(...) and with different shapes (thus Mann-Whitney test is not appropriate).

Yes, you've got it right. Technically speaking, the Mann-Whitney U test does not test for median, but whether one distribution is offset from the other, which is something subtly different. In particular, it makes this test sensitive to differences in distribution between groups. (see Morten W. Fagerland and Leiv Sandvik "The Wilcoxon-Mann-Whitney test under scrutiny", John Willey & Sons, 2009). These differences can translate e.g. into differences in variance or skewness. So this test, in contrast to the Welch test (the t-test with Welch modification), is not safe for when there is no variance homogeneity.

Improve this answer
$\endgroup$
3
  • $\begingroup$ @Miroslav For some reasons the stackexchange servers discarded my answer (and put only temporary edit). Fortunately I have backups, so here it is. $\endgroup$
    – Adam Ryczkowski
    Commented Oct 8, 2012 at 12:24
  • $\begingroup$ Thank You Adam, I saw only part of Your answer, nevertheless I understood it. Now, it is complete. Thanx again. $\endgroup$
    – sitems
    Commented Oct 8, 2012 at 12:36
  • 1
    $\begingroup$ I disagree in part with a number of the statements in this answer. The t-test, for example, does not have particularly good power-robustness against arbitrary choice of distribution shape and can have A.R.E. of 0 against the Wilcoxon-Mann-Whitney (it has okay level-robustness but power matters). The WMW isn't just a test for a shift alternative & does not actually require the same shape (though it's somewhat easier to interpret if that's the case). It's actually a test for whether P(X>Y) differs from P(Y>X), a clear and relatively simple concept even when distribution shapes differ. $\endgroup$
    – Glen_b
    Commented May 19, 2017 at 8:32
2
$\begingroup$

Perhaps you could use bootstrapping.

You have 100 points. If there is not a significant difference, then these 100 points together are representative of the entire distribution of values.

So, pool your samples, and draw (with replacement) two groups of 50 points from this sampling space. Measure the difference between the means and medians of these two groups of points. Repeat a few hundred times (which, since you have a computer, should take a few seconds at most!).

Now, measure the difference in mean and median between your two original samples . Are 95% of the bootstrapped differences smaller (or larger) than this difference? Then your difference is significant at a 95% confidence level.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Dear John, it looks very reasonably. Thank You for Your answer. $\endgroup$
    – sitems
    Commented Oct 8, 2012 at 9:18
  • $\begingroup$ @John, I'd say that the bootstrap is an overkill in this simple and classical situation. See my answer. $\endgroup$
    – Adam Ryczkowski
    Commented Oct 8, 2012 at 9:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.