15
$\begingroup$

I was wondering what are the criteria to use Kolmogorov-Smirnov, Cramer-von-Mises, and Anderson-Darling when comparing 2 ECDFS. I know the mathematics of how each differ, but if I have some ECDF data, how would I know which test is appropriate to use?

$\endgroup$
9
$\begingroup$

To cut a long story short: Anderson-Darling test is assumed to be more powerful than Kolmogorov-Smirnov test.

Have a glance on this article comparing various tests (of normality, but the results hold for comparing two distribudions) Power Comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors and Anderson-Darling Tests by Nornadiah Mohd Razali & Yap Bee Wah.

Anderson-Darling test is much more sensitive to the tails of distribution, whereas Kolmogorov-Smirnov test is more aware of the center of distribution.

To sum up, I would recommend you to use Anderson-Darling or eventually Cramer-von Misses test, to get much more powerful test.

$\endgroup$
  • 1
    $\begingroup$ Thanks for the info. What about CVM? When would that test be used? $\endgroup$ – Plinth Mar 13 '16 at 17:37
  • $\begingroup$ @Plinth Frankly speaking, if I were you, I would simply use AD as long as AD doesn't fail to meet your needs, that's just because my custom. But I don't say there's a general rule to prefer AD over CvM. In some cases, the significant effort is made to compare these 2 test in specific conditions like here (onlinelibrary.wiley.com/doi/10.1029/2004WR003204/full). It's up to you wheter it's worth to invest so much time to compare them and if think AD could be slightly better, but it's only my premonition. Maybe someone more experienced would speak out. $\endgroup$ – Adam Przedniczek Mar 13 '16 at 18:18
18
$\begingroup$

Each of the three tests have better power against different alternatives; but on the other hand, all three exhibit varying degrees of test bias in some situations.

Broadly speaking, the Anderson-Darling test has better power against fatter tails than specified and the Kolmogorov-Smirnov has more power against deviations in the middle, with Cramer-von Mises in between the two but somewhat more akin to the Kolmogorov-Smirnov in that respect.

The kinds of alternatives many people find to be of interest tend to be picked up more often by the Anderson-Darling and the Cramer-von Mises test but your particular needs may be different.

The Anderson-Darling tends to suffer worse bias problems overall (for hypothesis tests, bias means that there are some alternatives you're even less likely to reject than the null -- which is not what you want from an omnibus goodness of fit test -- but it seems to be difficult to avoid in realistic situations).

A number of power studies have been done which include an array of goodness of fit tests; generally for the alternatives that they consider, the Anderson-Darling tends to come out best most often --- but if you're testing uniformity and trying to pick up say a beta(2,2) alternative, none of them do well, and the Anderson Darling is the worst.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.