Evaluating goodness of fit for distributions (e.g. LogNorm, Gamma, …) with estimated parameters using KS tests (and R)

Currently I am trying to find a well-known distribution that fits to my positive skewed dataset (n=70) the best. First I used the fitdistrplus R package to estimate parameters for Gamma, Weibull, Lognormal and Exponential distributions (using Maximum Likelihood estimation, though I am unsure if MLE is the best choice with 70 observations (better one?)).

In the second step, I selected the model with the smallest AIC. But of course the model should also pass a goodness of fit test. The first idea was simply using an Kolmogorv-Smirnov test with the estimated parameter, but this doesn't seem to be a good idea since KS-Tests with estimated parameters lead to more or less useless p values.

During my search on the web, I stumbled over Greg Snows' suggestion and apart from that over this page which describes an interesting monte carlo approach (from Clauset et al.). An exemplary adapted R code sample which uses the fitdistrplus package for maximum likelihood estimation for the log norm distribution looks as follows:

lognormal = function(d, limit=2500) {
# MLE for lognormal distribution
fit <- fitdist(d,"lnorm", method="mle")

# compute KS statistic
t = ks.test(d, "plnorm", meanlog = fit$estimate["meanlog"], sdlog = fit$estimate["sdlog"]);

# compute p-value
count = 0;
for (i in 1:limit) {
syn = rlnorm(length(d), meanlog = fit$estimate["meanlog"], sdlog = fit$estimate["sdlog"]);
fit2 <- fitdist(syn, "lnorm", method="mle")
t2 = ks.test(syn, "plnorm", meanlog = fit2$estimate["meanlog"], sdlog = fit2$estimate["sdlog"]);
if(t2$stat >= t$stat) {count = count + 1};
}

return(list(meanlog = fit$estimate["meanlog"], sdlog = fit$estimate["sdlog"], stat = t$stat, p = count/limit, KSp = t$p));
}

What I am currently asking me (and you) is, does this approach makes sense with respect to the small sample size (or should I use moment/... estimators or is MLE ok) and is the way the goodness of fit is tested suitable?

• Did you decide to go with this method to test for goodness of fit? I'm looking at using the same approach, albeit with a larger n. – fmark May 16 '14 at 4:49
• "But of course the model should also pass a goodness of fit test." -- why "of course"? It's very unlikely any of the distribution families contain the actual population distribution for these data. – Glen_b Jul 5 '14 at 10:29