Generate Example data for which it is difficult to distinguish between Gamma, Weibull and log-normal fit using R? I'm trying to generate a data set, as a demonstration case, to show a case in which it is difficult to distinguish between Gamma, Weibull and log-normal distribution. To do this I generate some data:
x = rlnorm(500,-1.5,0.5)
grid = seq(-1,3,.01)
h=hist(x, breaks=20, freq = FALSE, col = "grey")

Is there a best fit routine within R that will allow me to determine the optimal parameters for the three distributions? And is there a better data set (other than rlnorm) which will show this case more clearly? I need a case in which the three distributions are very close.
 A: One way to search for such a distribution (and then simulate data from it) would be to start with expressions for Kullback-Leibler distances between them, and search for parameter values that make all of them small.  Se my answer to 
Intuition on the Kullback-Leibler (KL) Divergence
Using the ideas from above referenced post, I will show how to find parameter values giving samples which will be difficult to discriminate between these three distributions.  First some R code for calculating the KL divergence:
KL  <-  function(p, q, ppar=c(1, 1), qpar=c(1, 1), lower=0,  upper=+Inf)  {
    integrate(function(x) p(x, ppar[1], ppar[2]) * (
        p(x, ppar[1], ppar[2], log=TRUE)-q(x, qpar[1], qpar[2], log=TRUE) ),
        lower=lower, upper=upper)$value }

One example of use: 
 KL(dweibull, dgamma, ppar=c(1, 1), qpar=c(1, 1))
[1] 0
KL(dweibull, dgamma, ppar=c(1, 1), qpar=c(2, 1))
[1] 0.5772157

and indeed, if plotting the first pair of densities, they cannot be distinguished in the plot. 
Then we use optim to find parameter values minimizing KL divergence:
minKL  <-  function(p, q, ppar=c(1, 1), qparstart=c(1, 1),
                    qparlower=c(0, 0), qparupper=c(Inf, Inf), ...)   {
    ### ... used for controls to optim 
    res  <-  optim(qparstart, function(qpar) KL(p, q, ppar, qpar),
                   method="L-BFGS-B", lower=qparlower, upper=qparupper, ...)
    return(list(res$par, KL_value=res$value))
}

This function is not very robust, and need starting values for qpar close enough that the integral converges.  The first argument p is kept fixed, and then the parameter values for the second argument q is found to minimize $\text{KL}(p \| q)$.  This is finding q such that the test of $H_0\colon P$ against $H_A\colon Q$ has low power, that is, the discrimination is difficult.
One example of use is:
 minKL(dweibull, dgamma, ppar=c(1, 1), qparstart=c(1, 1))
[[1]]
[1] 1.000211 1.000000

$KL_value
[1] -1.420418e-07

and in this case, indeed, the densities cannot be distinguished when plotted (try!).  An example where the minimum distance is larger is: 
One example where the divergence is larger is 
minKL(dweibull, dgamma, ppar=c(5, 1), qparstart=c(5, 1))
[[1]]
[1] 16.79337 18.29007

$KL_value
[1] 0.05059095

and a plot of the densities is below:

I will leave the calculation in the other direction as an exercise.  To find a set of parameters that give distributions which is close for all three pairs, in both directions, one might calculate the six integrals, take the max, and minimize that.  
It could be more useful to find explicit expressions for this distances, I will not try that today!
