41
$\begingroup$

I have an experimentally observed distribution that looks very similar to a gamma or lognormal distribution. I've read that the lognormal distribution is the maximum entropy probability distribution for a random variate $X$ for which the mean and variance of $\ln(X)$ are fixed. Does the gamma distribution have any similar properties?

$\endgroup$
6
  • 2
    $\begingroup$ Why would such a property be of any value in deciding which would be an appropriate model? $\endgroup$ – Glen_b Oct 9 '13 at 21:57
  • $\begingroup$ @Glen_b I'm still a beginner when it comes to statistics so my knowledge is pretty basic. Looking at the plots of gamma and lognormal distributions, qualitatively they look very similar. I'm looking for quantitative differences between the two. For instance, what are some examples of physical applications where gamma or lognormal distributions occur? $\endgroup$ – OSE Oct 9 '13 at 22:06
  • $\begingroup$ In reality, likely neither ever actually occurs; they're extraordinarily simple models which are sometimes useful (if rough) approximations of reality. I will post an answer that discusses some qualitative differences. $\endgroup$ – Glen_b Oct 9 '13 at 22:31
  • 1
    $\begingroup$ @glen_b: the reason is that if you're measuring only those statistics then the minimum assumptive distribution is uniquely the exponential family distribution with those sufficient statistics. Whereas any distribution might be a poor model of reality, if one is not free to choose which measurements are taken, then this is an excellent way of choosing a model. $\endgroup$ – Neil G Oct 10 '13 at 9:04
  • 1
    $\begingroup$ @Glen_b I guess the lognormal distribution should appear in some physical situations because of the CLT. $\endgroup$ – Stéphane Laurent Oct 10 '13 at 9:06
38
$\begingroup$

As for qualitative differences, the lognormal and gamma are, as you say, quite similar.

Indeed, in practice they're often used to model the same phenomena (some people will use a gamma where others use a lognormal). They are both, for example, constant-coefficient-of-variation models (the CV for the lognormal is $\sqrt{e^{\sigma^2} -1}$, for the gamma it's $1/\sqrt \alpha$).

[How can it be constant if it depends on a parameter, you ask? It applies when you model the scale (location for the log scale); for the lognormal, $\mu$ acts as a scale parameter, while for the gamma, the scale is the parameter that isn't the shape parameter (or its reciprocal if you use the shape-rate parameterization). I'll call the scale parameter for the gamma distribution $\beta$. Gamma GLMs model the mean ($\mu=\alpha\beta$) while holding $\alpha$ constant; in that case $\mu$ is also a scale parameter. A model with varying $\mu$ and constant $\alpha$ or $\sigma$ respectively will have constant CV.]

You might find it instructive to look at the density of their logs, which often shows a very clear difference.

The log of a lognormal random variable is ... normal. It's symmetric.

The log of a gamma random variable is left-skew. Depending on the value of the shape parameter, it may be quite skew or nearly symmetric.

Here's an example, with both lognormal and gamma having mean 1 and variance 1/4. The top plot shows the densities (gamma in green, lognormal in blue), and the lower one shows the densities of the logs:

gamma and lognormal, densitiy and density of log

(Plotting the log of the density of the logs is also useful. That is, taking a log-scale on the y-axis above)

This difference implies that the gamma has more of a tail on the left, and less of a tail on the right; the far right tail of the lognormal is heavier and its left tail lighter. And indeed, if you look at the skewness, of the lognormal and gamma, for a given coefficient of variation, the lognormal is more right skew ($\text{CV}^3+3\text{CV}$) than the gamma ($2\text{CV}$).

$\endgroup$
4
  • $\begingroup$ +1. Do you know if there is a closed formula for the skewness of the log of gamma? For lognormal, the skewness of log is clearly zero, and I am wondering if there is some expression for the gamma. Wikipedia gives formulas for the mean and the variance of log(gamma), but not for the skewness. $\endgroup$ – amoeba Nov 8 '17 at 15:43
  • $\begingroup$ Gradshteyn & Ryzhik (sect 4.358, 7th ed) list explicit closed forms for $\int_0^\infty x^{\nu-1}e^{-\mu x}(\ln x)^p dx$ for $p=2,3,4$ while the $p=1$ case is done in 4.352 (assuming you regard expressions in $\Gamma, \psi$ and $\zeta$ functions as closed form) -- from which it is definitely doable up to kurtosis; they give the integral for all $p$ as a derivative of a gamma function so presumably it's feasible to go higher. So skewness is certainly doable but not especially "neat". If you want to pursue it I could give you the integrals. $\endgroup$ – Glen_b Nov 9 '17 at 2:16
  • $\begingroup$ However we don't need to evaluate the skewness in order to discern its sign. Examining the log of the density of the logs should suffice to establish that because one side clearly dominates the other. $\endgroup$ – Glen_b Nov 9 '17 at 4:03
  • $\begingroup$ Thanks Glen. I decided to post it as a new question: stats.stackexchange.com/questions/312803. I spent some time searching for a ready answer but could not find any, so it might be valuable for the future to have it written down somewhere where it's easy to find. It might be a somewhat better fit for Math.SE, but I'd prefer to have it here, to be honest. $\endgroup$ – amoeba Nov 9 '17 at 10:11
13
$\begingroup$

Yes, the gamma distribution is the maximum entropy distribution for which the mean $E(X)$ and mean-log $E(\log X)$ are fixed. As with all exponential family distributions, it is the unique maximum entropy distribution for a fixed expected sufficient statistic.

To answer your question about physical processes that generate these distributions: The lognormal distribution arises when the logarithm of X is normally distributed, for example, if X is the product of very many small factors. If X is gamma distributed, it is the sum of many exponentially-distributed variates. For example, the waiting time for many events of a Poisson process.

$\endgroup$
1
  • 5
    $\begingroup$ No need of "many" exponential variates to be Gamma. $\endgroup$ – Stéphane Laurent Oct 10 '13 at 9:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.