1
$\begingroup$

I have data that is officially not normally distributed (Kolmogorov-Smirnov test < .000 BUT N=around 1200 so K-S not really reliable). In plot I can see that form looks good but negatively skewed. Homogeneity of variance assumption is more certainly not met though (Levene's < .001) SO -> Mann-Whitney U test (1 cont. dep.var. with 2 level indep. var between subjects).

Using a simple formula I calculated effect-sizes from the (SPSS) output from this test. I did the same with t-Test results (other formula of course).

My question is how can these results differ so much (changed to explained variance) the estimates (for the same effect) are (example) 15% (based on N & z) and 45% (based of t & df)?

Update (at comment Horst Grünbusch (merci)): Let's assume for a second that the Shapiro Wilk is <.05, Could I then report the effect size based on Mann-Whitney U (see above) disregarding the huge difference with the ('inapproriat because for parametric') t-Test based effect-size?

Improve this question
$\endgroup$
5
  • 1
    $\begingroup$ The Kolmogorov-Smirnov test looks if the data follow a certain normal distribution. How did you specify $\mu$ and $\sigma$ for that purpose? Generally, in order to test if the data come from some normal distribution, there is the Shapiro-Wilk-test. More generally, Goodness-of-fit tests are not appropriate to choose subsequent test statistics. $\endgroup$
    – Horst Grünbusch
    Commented Sep 26, 2014 at 10:39
  • $\begingroup$ More important question: How did you calculate/understand the effect size for the Wilcoxon-Mann-Whitney test? $\endgroup$
    – Horst Grünbusch
    Commented Sep 26, 2014 at 10:42
  • $\begingroup$ Thank you, I divided the z value by the square root of N (for the Mann-Whitney U effect size..) $\endgroup$
    – Steven B. Peutz
    Commented Sep 26, 2014 at 10:59
  • $\begingroup$ "Kolmogorov-Smirnov test < .000 BUT N=around 1200 so K-S not really reliable" ... what does this mean? In what sense is it "not reliable"? $\endgroup$
    – Glen_b
    Commented Sep 26, 2014 at 13:00
  • $\begingroup$ Kolmogorov Smirnov (as well as Shapiro Wilk) are based on NHST and so they tend to be overly strict in large samples (because more easily signicant if large N) due to their larger 'power'(as with NHST). $\endgroup$
    – Steven B. Peutz
    Commented Sep 26, 2014 at 13:10

1 Answer 1

Reset to default
2
$\begingroup$

What you have considered to be the effect of the Wilcoxon(-Mann-Whitney) test is actually the test statistic itself (up to a known constant). It is not an effect estimator.

The effect married with the WMW test is the "Common language effect", also called relative nonparametric effect. This effect is a probability: If you take randomly one subject from each population, this effect is the probability that the subject from the first population has a larger observed value than the subject from the second population. An unbiased and consistent estimator for this probability is the relative frequency that this happens among pairs of subjects from both samples.

If you want to compare WMW and $t$-test, you have to observe that the relative nonparametric effect cannot be compared directly to the normal distribution's effect. Generally, the relative effect fulfills $$P(X_1 > X_2) = \int F_1(x)\mathrm{d}F_2(x),$$ so if you want $F_1$ and $F_2$ to be normal distributions, you have to fix both variances. Then you can see how powerful the $t$-test is w.r.t. the nonparametric relative effect.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.