How to determine the best distribution to fit my data

Question

Currently I'm doing research about distribution. I am trying to find the best distribution fit to some continuous data. I'm using 3 goodness of fit tests, namely Kolmogorov-Smirnov (KS), Anderson-Darling (AD) and Cramér-von Mises (CVM):

library("fitdistrplus")
library("actuar")
x<-c(1.89,5.73,5.27,6.77,1.08,7.5,4.24,1.12,2.68,1.29,2.68,1.26,1.1,1.19,4.16,
1.71,5.7,2.88,5.43,12.88,1.35,1.2,1.07,1.03,1.39,1.72,4.76,1.45,1.37,5.91,9.59,
4.36,5.15,1.85,2.82,8.58,1,6.93,1.48,6.23,12.42,1.22,7.02,2.73,7.11,1.93,
5.68,1.03,1.26,1.07,1.22)

  fg <- fitdist(x, "gamma")
  fln <- fitdist(x, "lnorm")
  fw <- fitdist(x, "weibull")
  fe <- fitdist(x, "exp")
  fll <- fitdist(x, "llogis")

 gofstat(list(fg,fln,fw,fe,fll),fitnames=c("gamma","lnorm","weibull","exp","llogis"))

output>>>

Goodness-of-fit statistics
                                 gamma     lnorm   weibull       exp    llogis
Kolmogorov-Smirnov statistic 0.1847511 0.1691589  0.1769022  0.2370566 *0.1533049*
Cramer-von Mises statistic   0.3724219 0.3511080  0.3340045  0.3419064 *0.3277217*
Anderson-Darling statistic   2.1727043 2.0787734 *1.9974584* 2.3348410  2.0238724

Goodness-of-fit criteria
                                  gamma    lnorm  weibull      exp   llogis
Akaike's Information Criterion 231.2868 226.0944 233.3155 237.3364 231.5575
Bayesian Information Criterion 235.1505 229.9580 237.1791 239.2682 235.4211

If 2 out of 3 tests prefer for example log-logistic, and only one prefers the Weibull, the log-logistic is the best. Am I right?

How about if each test prefers a different distribution, which one is the best?

Answer 1

I think this exercise is not an especially good approach and your interpretation of the output somewhat compounds the issues.

1. You ask about what's "best" without saying what you want to be best at. What is it you need this distribution choice to achieve? What's it for?? That issue alone makes your question effectively unanswerable. I will proceed by rejecting the premise of the question, which might hopefully lead you to a more fruitful line of thinking.

2. None of those distributions you consider will actually be correct. The question is really whether any of them are good enough for some particular purpose -- which requires detailed examination of that purpose as well as the particular properties of those distributions. It may be that some distributional aspects are relatively unimportant and others matter a lot, in which case you should be focusing your attention on the things that matter for your application.

3. Why choose among that shopping list of distributions rather than some other collection? What makes those better than something else?

4. Why use that shopping list of goodness of fit tests rather than some other collection? What makes those better than something else?

5. Exponential is a special case of gamma and Weibull (but with one fewer parameter than either of them). It will always fit worse than the gamma or the Weibull if you don't account for the effect of having an additional parameter. So the way you're looking at it, considering the exponential would be a waste of time.

6. Different goodness of fit statistics measure different aspects of the misfit between data and distribution. It's not a collection of "equally good" votes, but measures whose relative ability to discern differences varies against different sets of alternatives. Better to understand what they're looking at and choose a good one, or design a measure that measures more precisely what aspects of the distribution will matter for your problem.

In short, begin with considering what you need this for - and indeed whether you need a specific simple parametric distributional form at all, rather than something else. Then proceed to consider as far as you can what aspects of the distribution matter most (e.g. is it mostly the upper tail? the mean? etc. In some cases you might even need a simulation study even to figure out what's important).