I am looking at fitting distributions to data (with a particular focus on the tail) and am leaning towards Anderson-Darling tests rather than Kolmogorov-Smirnov. What do you think are the relative merits of these or other tests for fit (e.g. Cramer-von Mises)?
$\begingroup$
$\endgroup$
0
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
I have been told many times that the Anderson Darling (AD) test is much better than the Kolmogorov-Smirnov (KS) one because AD does a better job at fitting the tails of the distribution. KS is only good at fitting the mid-range of the distribution; but, is not better than AD even in this regard. I think the main advantage of the KS test is its very intuitive visual interpretation (fitting of the respective cumulative distributions). Because of the KS easy visual and intuitive interpretation it has become dominant in certain specialties such as credit scoring models within the financial service industry. But, more visually intuitive does not mean better.
When using Monte Carlo simulation models that automatically fit a statistical distribution to a data set; their respective software manuals typically recommend leaning more on the AD than the KS test for the reason mentioned above (fits the tails better).
$\begingroup$
$\endgroup$
2
I'm not sure about these tests, so this answer may be off-topic. Apologies if so. But, are you sure that you want a test? It really depends on what the purpose of the exercise is. Why are you fitting the distributions to the data, and what will you do with the fitted distributions afterward?
If you want to know what distribution fits best just because you're interested, then a test may help.
On the other hand, if you want to actually do something with the distribution, then you'd be better off developing a loss function based on your intentions, and using the distribution that gives you the most satisfactory value for the loss function.
It sounds to me from your description (particular focus on the tail) that you want to actually do something with the distribution. If so, it's hard for me to imagine a situation where an existing test will provide better guidance than comparing the effects of the fitted distributions in situ, somehow.
-
$\begingroup$ I agree completely with what you're saying, but the reason for looking at tests here is to satisfy others. The situation is modelling possible extreme operational losses bassed on historical loss experience and the regulator needs to be convinced that the choice of distribution is supported by the data. What the regulator thinks is reasonable and what the business thinks is reasonable for the results can differ quite a bit! Using a (reasonably) standard statistical test may provide a somewhat independent approach to the justifying a particular choice. $\endgroup$ Jul 21, 2010 at 5:40
-
$\begingroup$ Then using a 'distance between distribution test' (like chi square or Kolmogorov-Smirnov or ... is a better idea because it is easely understood by the end user. $\endgroup$ Jul 21, 2010 at 6:57
$\begingroup$
$\endgroup$
I think that my question is subsumed by this more general discussion: Motivation for Kolmogorov distance between distributions
