I was wondering what are the criteria to use Kolmogorov-Smirnov, Cramer-von-Mises, and Anderson-Darling when comparing 2 ECDFS. I know the mathematics of how each differ, but if I have some ECDF data, how would I know which test is appropriate to use?
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
2
To cut a long story short: Anderson-Darling test is assumed to be more powerful than Kolmogorov-Smirnov test.
Have a glance on this article comparing various tests (of normality, but the results hold for comparing two distribudions) Power Comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling Tests by Nornadiah Mohd Razali & Yap Bee Wah.
Anderson-Darling test is much more sensitive to the tails of distribution, whereas Kolmogorov-Smirnov test is more aware of the center of distribution.
To sum up, I would recommend you to use Anderson-Darling or eventually Cramer-von Misses test, to get much more powerful test.
answered Mar 13, 2016 at 15:52
-
2$\begingroup$ Thanks for the info. What about CVM? When would that test be used? $\endgroup$– PlinthMar 13, 2016 at 17:37
-
$\begingroup$ @Plinth Frankly speaking, if I were you, I would simply use AD as long as AD doesn't fail to meet your needs, that's just because my custom. But I don't say there's a general rule to prefer AD over CvM. In some cases, the significant effort is made to compare these 2 test in specific conditions like here (onlinelibrary.wiley.com/doi/10.1029/2004WR003204/full). It's up to you wheter it's worth to invest so much time to compare them and if think AD could be slightly better, but it's only my premonition. Maybe someone more experienced would speak out. $\endgroup$ Mar 13, 2016 at 18:18
$\begingroup$
$\endgroup$
0
Each of the three tests have better power against different alternatives; but on the other hand, all three exhibit varying degrees of test bias in some situations.
Broadly speaking, the Anderson-Darling test has better power against fatter tails than specified and the Kolmogorov-Smirnov has more power against deviations in the middle, with Cramer-von Mises in between the two but somewhat more akin to the Kolmogorov-Smirnov in that respect.
The kinds of alternatives many people find to be of interest tend to be picked up more often by the Anderson-Darling and the Cramer-von Mises test but your particular needs may be different.
The Anderson-Darling tends to suffer worse bias problems overall (for hypothesis tests, bias means that there are some alternatives you're even less likely to reject than the null -- which is not what you want from an omnibus goodness of fit test -- but it seems to be difficult to avoid in realistic situations).
A number of power studies have been done which include an array of goodness of fit tests; generally for the alternatives that they consider, the Anderson-Darling tends to come out best most often --- but if you're testing uniformity and trying to pick up say a beta(2,2) alternative, none of them do well, and the Anderson Darling is the worst.
