Understanding how to apply goodness of fit tests when parameters of a continuous non-normal distribution have been fitted to the data

Question

I have some data and wish to fit several distributions to it, many of which are compound and/or complex. I'd like to know whether a given family of distributions is appropriate which I can apparently do via Anderson-Darling.

Wikipedia:
Essentially the same test statistic can be used in the test of fit of a family of distributions, but then it must be compared against the critical values appropriate to that family of theoretical distributions and dependent also on the method used for parameter estimation.

How do I do this for distributions other than normal (the worked example)? Directions to a paper or book are very much appreciated if this is common knowledge but I'm struggling to find it.

Answer 1

The classic reference for goodness of fit tests is D'Agostino & Stephens (eds) (1986) Goodness-of-Fit Techniques, Marcel Dekker Inc.

(That book doesn't cover smooth tests at all, though -- for that you'd need the little book by Rayner & Best. Fortunately, the Anderson-Darling is in the class of ECDF tests which is well covered by D'Agostino & Stephens' book)

The relevant parts of the book discuss parameter estimation in the Anderson-Darling (to my recollection those parts are written by Stephens; he also wrote a number of relevant papers). A number of common distributions are discussed.

Note that if you're fitting multiple distributions and choosing one that fits best, the properties of the test will also be impacted by that process in a similar sense to optimizing the fit over parameters within a family, Imagine for example if you were in the situation that all the distribution families you considered fell into an even larger class indexed by some index-parameter(s) - you'd then doing a form of discrete optimization over those index parameters, and clearly that would impact the inference relating to goodness of fit for the same reason as before. This effect is still there whether or not you can formally write them all as being in some super-class.