1
$\begingroup$

I need to test to compare the means of two samples, one with size 21 and the other with size 47. Histograms show Sample 1 is skewed to the right while Sample 2 has a bell symmetrical shape. Descriptive statistics will be found below. I tested for normality with Shapiro-Wilk any way, rejecting the null hypothesis for either, so that will confirm they are not normal. I understand t-test will not be appropriate to this case. I moved to non-parametric and used Wilcoxon rank sum test and got p-value 0.2395 which will indicate there is no evidence to think the means are different (they are equal).

Is it correct to use Wilcoxon rank sum in this case where the shape of both samples is different? Am I missing something? Or with this information can I state that there is no difference between the means? Is it necessary to do something else like permutations?

Some statistics...

Sample 1
Shapiro-Wilk p-value = 0.000
average = 0.0270
sd = 0.0892
kurtosis 10.4
skewness 3.2
n = 21

Sample 2
Shapiro-Wilk p-value = 0.0366
average = 0.0367
sd = 0.0752
kurtosis 0
skewness 0.6
n = 46
Improve this question
$\endgroup$
3
  • $\begingroup$ I've left your "not rejected" in respect of P-values < 0.05. What makes you think Wilcoxon compares means? $\endgroup$
    – Nick Cox
    Feb 25, 2014 at 0:17
  • $\begingroup$ Whether the Wilcoxon is approprate depends on the precise hypothesis you are testing. See the discussion here, starting from "There are two main ways to look at the Wilcoxon-Mann-Whitney hypothesis test" (toward the bottom of the answer, roughly the last two screens on my laptop) $\endgroup$
    – Glen_b
    Feb 25, 2014 at 0:25
  • $\begingroup$ Thanks Nick Cox, that was a typo. I Should have said "rejecting the null hypothesis for either, so that will confirm they are not normal". I already edited my question to correct it. $\endgroup$
    – Gina
    Feb 25, 2014 at 2:01

1 Answer 1

Reset to default
1
$\begingroup$

There are several problems with what you have written:

I need to test to compare the mean of two datasets: one with 21 samples and the other with 47. Histogram shows Dataset 1 is skewed to the right when Dataset 2 has a bell symmetrical shape. Descriptive statistics will be found below. I tested for normality with Shapiro-Wilks anyway not rejecting the null hypothesis for any of them, so that will confirm they are not normal.

If you failed to reject the null in SW then you have not confirmed that they are not normal; that would be the conclusion if you had rejected the null.

I understand t-test will not be appropriate to this case.

The t-test is reasonably robust to violations of normality if the variances are not too different and the sample sizes not too different

I moved to non-parametric and used Wilcoxon rank sum test and got p-value 0.2395 which will indicate there is no evidence to think the means are different (they are equal)

The Wilcoxon test does not exactly compare means, it compares mean ranks.

A p-value of 0.24 does not indicate that there is no evidence to think the mean ranks are different; it indicates that there is not enough evidence to conclude they are different at whatever level of significane. Three is, however, some evidence that they are different.

Is it correct to use Wilcoxon rank sum in this case where the shape of both datasets look different? Am I missing something? or with this information, Can I state that there is no difference in the means? Is it necessary to do something else like permutations? I'll really appreciate your advice. Thank you.

You cannot state there is no difference in the mean ranks; you can state that there is not sufficient evidence to conclude they are different.

If you want to compare mean ranks, then what you have done is fine (if you correct your conclusions) but if you want to compare something else, you will need a different test; maybe permutations.

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ Peter I think that the $t$-test is not as robust to non-normality as most commonly believe. In effect, the central limit theorem only truly applies if one knows $\sigma^2$, which is seldom the case. The need for estimating $\sigma^2$ implies that people use the $t$-distribution for $P$-values and confidence limits, and the tiniest bit of non-normality can make the $t$ ratio far from a $t$ distribution. $\endgroup$
    – Frank Harrell
    Feb 25, 2014 at 13:27
  • 1
    $\begingroup$ @FrankHarrell I've seen a variety of Monte Carlo studies showing different things for different violations of the assumptions. But, sure, to be safe, it's better to not use t when the assumptions are violated. But, how much violated? If we test sensitively enough, the assumptions will always be wrong. $\endgroup$
    – Peter Flom
    Feb 25, 2014 at 13:49
  • 1
    $\begingroup$ Rand Wilcox has published examples where the non-normality is not detectable from a density plot but the impact on $P$-values from the $t$-test is large. One way I think of it is that with extreme skewness, $\sigma$ is not even a viable parameter for summarizing dispersion. $\endgroup$
    – Frank Harrell
    Feb 25, 2014 at 16:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.