is there a way of calculating an effect size for the Kolmogorov-Smirnov Z statistic (in SPSS or by hand)? Or should I stick to the Mann-Whitney test, even though my group sizes are less than n=25?
$\begingroup$
$\endgroup$
7
asked Mar 2, 2011 at 0:54
-
$\begingroup$ First of all, I wouldn't place much trust in the K-S test with samples <25 or >300. But otherwise, interesting: how to quantify on a unidimensional, 0-to-1 scale? On the one hand, it's not hard to imagine that when a distribution shows a perfect fit to the hypothesized one, the effect size would be 1. But what about radically poor fits--how would one distinguish between the poor fits to the normal of, say, a U-shaped distribution, a uniform, or some multi-humped polynomial one? Also, there's a question of scale: it seems Z would have to be infinite for the effect size to be 0. $\endgroup$– rolando2Commented Mar 3, 2011 at 18:10
-
$\begingroup$ please take the tour and read the help -- links are at the bottom left of this page $\endgroup$– Glen_bCommented Nov 20, 2017 at 0:34
-
$\begingroup$ @rolando2 why you wouldn't trust K-S test for sample sizes >300? at least in R I see it working as expected even with sample sizes of 10000 or more... $\endgroup$– Hubert KarioCommented Jun 28, 2020 at 17:50
-
$\begingroup$ @HubertKario From experience with simulation. Samples drawn from a normal population distribution are disproportionately judged non-normal by the K-S when N is large. $\endgroup$– rolando2Commented Jun 28, 2020 at 18:22
-
$\begingroup$ @rolando2 sorry, I can't replicate that, I'm getting expected false positive rate of 5% at alpha of 0.05 both when comparing two sample sets and a sample set to a distribution: gist.github.com/tomato42/c2ff6be65bbd1fbed83c31a5089cd06b Am I doing something wrong? $\endgroup$– Hubert KarioCommented Jun 28, 2020 at 18:37
|
Show 2 more comments
1 Answer
$\begingroup$
$\endgroup$
3
Yes. $D = Z/\sqrt{n}$ for the one-sample test. $D = Z/\sqrt{\frac{n_1 n_2}{n_1 + n_2}}$ for the two-sample test. $D$ should also be the "Most Extreme Differences - Absolute" entry in the output graphic (double-click the table shown in the SPSS output viewer). $Z$ might be labeled "Test Statistic," "Kolmogorov-Smirnov Z," or something else depending on which test and version of SPSS you're using.
It depends. Mann-Whitney tests for a difference in the central tendencies by comparing average ranks; K-S tests for a difference in distributions by comparing the maximum difference in empirical cumulative distribution functions. If you expect strong shape differences, such as only low and high values in one group but middle values for the other group (this would be atypical for most data), K-S is a better choice. If you expect just a location shift, Mann-Whitney is more powerful.
-
1$\begingroup$ Your answer to #1 converges to zero as the sample size grows, so what justifies interpreting it as an effect size? $\endgroup$– whuber ♦Commented Mar 3, 2011 at 19:33
-
$\begingroup$ $D$ is defined as the supremum of absolute differences in the two ECDFs; $Z$ is defined as $\sqrt{n_1 n_2 / (n_1+n_2)} D $. I was just explaining how to compute $D$ from SPSS output. Hope that clarifies. $\endgroup$ Commented Mar 3, 2011 at 20:24
-
$\begingroup$ Are there any rule of thumb concerning when they are big, medium, or small? $\endgroup$ Commented Sep 19, 2014 at 11:28