4
$\begingroup$

I have data from 4 populations : A (n=16), B (n=16), C (n=14) and D (n=16).

Data from A and B are not normally distributed. Data from C and D are normally distributed. All data passed the Levene's test for equality of variance. Kruskal-Wallis test does not show any differences between group.

The independent Mann-Whitney test gave significant results when comparing A and C, but not when comparing A and D, although the median differences between A and D were greater than those between A and C.

I am not familiar with stat, and very confused by the results. Could you please help me to understand ?

Improve this question
$\endgroup$
2
  • 6
    $\begingroup$ What is your research question? $\endgroup$
    – user2974951
    Commented Dec 4, 2023 at 12:30
  • 5
    $\begingroup$ Three mistaken premises in the question: 1. "C and D are normally distributed" - they almost certainly aren't (you can probably tell that for certain without a test). You failed to reject normality, but you'd equally fail to reject many other things (and with small sample size, even quite different things). C&D can't have each of the distributions that you would fail to reject, since they're mutually exclusive possibilities. 2. Mann-Whitney does not compare medians (per answers to many dozens of questions on site, see "Related"->). 3. Larger differences don't necessarily mean lower p-values $\endgroup$
    – Glen_b
    Commented Dec 4, 2023 at 17:16

3 Answers 3

Reset to default
11
$\begingroup$

Welcome to CV.

First, when you have N = 16, any test of normality is likely to be fairly useless. How did you determine normality?

Second, while we can't tell exactly what is going on without the data, Mann Whitney U is not a test of medians. To quote Wikipedia:

In statistics, the Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW/MWU), Wilcoxon rank-sum test, or Wilcoxon–Mann–Whitney test) is a nonparametric test of the null hypothesis that, for randomly selected values X and Y from two > populations, the probability of X being greater than Y is equal to the probability of Y being greater than X.

If you want to see if the medians are different, one choice is Mood's median test, but it has relatively low power.

Improve this answer
$\endgroup$
11
$\begingroup$

A couple of points to supplement what Frank and Peter wrote.

The normality assumption is that your data are sampled from distributions (or populations) that are (approximately) normal. It makes no sense to ask about the normality of a particular data set, only about the population the data are sampled from.

If you really think that datasets A and B come from populations with differently shaped distributions than C and D, aren't you done? What is left to test?

If you showed us a scatter graph of your data, with some scientific context, you might get more helpful suggestions.

Improve this answer
$\endgroup$
10
$\begingroup$

As Peter implied, trusting the data to inform you about normality is trusting them too much. And it implies that you think that the Wilcoxon or Kruskal-Wallis tests don’t work well with normal data when in fact they are 0.95 efficient as parametric tests when normality holds. Similarly a simple hypothesis test on the variance ratio has problems.

I would use a generalization of Wilcoxon-Kruskal-Wallis that allows one to test or compute confidence intervals for any contrast of interest, and also to adjust for covariates while doing that. Such a generalization is ordinal semiparametric regression models such as the proportional odds model.

An alternative is the Bayesian $t$-test which allows the degree of non-normality and the variance ratio to move with the data while taking into account all uncertainties in constructing uncertainty intervals.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.