I have an experimentally observed distribution that looks very similar to a gamma or lognormal distribution. I've read that the lognormal distribution is the maximum entropy probability distribution for a random variate $X$ for which the mean and variance of $\ln(X)$ are fixed. Does the gamma distribution have any similar properties?
$\begingroup$
$\endgroup$
6
-
2$\begingroup$ Why would such a property be of any value in deciding which would be an appropriate model? $\endgroup$– Glen_bCommented Oct 9, 2013 at 21:57
-
$\begingroup$ @Glen_b I'm still a beginner when it comes to statistics so my knowledge is pretty basic. Looking at the plots of gamma and lognormal distributions, qualitatively they look very similar. I'm looking for quantitative differences between the two. For instance, what are some examples of physical applications where gamma or lognormal distributions occur? $\endgroup$– OSECommented Oct 9, 2013 at 22:06
-
1$\begingroup$ @glen_b: the reason is that if you're measuring only those statistics then the minimum assumptive distribution is uniquely the exponential family distribution with those sufficient statistics. Whereas any distribution might be a poor model of reality, if one is not free to choose which measurements are taken, then this is an excellent way of choosing a model. $\endgroup$– Neil GCommented Oct 10, 2013 at 9:04
-
2$\begingroup$ @Glen_b I guess the lognormal distribution should appear in some physical situations because of the CLT. $\endgroup$– Stéphane LaurentCommented Oct 10, 2013 at 9:06
-
1$\begingroup$ Since we deal with finite rather than infinite samples, that case (a product many terms suggesting potential application of CLT to the logs) would be one of the cases where where might sometimes expect to see those "useful approximations of reality" I previously mentioned. $\endgroup$– Glen_bCommented Sep 14, 2016 at 5:42
|
Show 1 more comment
2 Answers
$\begingroup$
$\endgroup$
4
As for qualitative differences, the lognormal and gamma are, as you say, quite similar.
Indeed, in practice they're often used to model the same phenomena (some people will use a gamma where others use a lognormal). They are both, for example, constant-coefficient-of-variation models when the shape is held constant and the mean (in effect a scale parameter) varies: the CV for the lognormal is $\sqrt{e^{\sigma^2} -1}$, for the gamma it's $1/\sqrt \alpha$.
[How can it be constant if it depends on a parameter, you ask? It applies when you model the scale (location for the log scale); for the lognormal, the $\mu$ parameter acts as the log of a scale parameter, while for the gamma, the scale is the parameter that isn't the shape parameter (or its reciprocal if you use the shape-rate parameterization). I'll call the scale parameter for the gamma distribution $\beta$. Gamma GLMs model the mean ($\mu=\alpha\beta$) while holding $\alpha$ constant; in that case $\mu$ is also a scale parameter. A model with varying $\mu$ and constant $\alpha$ or $\sigma$ respectively will have constant CV.]
You might find it instructive to look at the density of their logs, which often shows a very clear difference.
The log of a lognormal random variable is ... normal. It's symmetric.
The log of a gamma random variable is left-skew. Depending on the value of the shape parameter, it may be quite skew or nearly symmetric.
Here's an example, with both lognormal and gamma having mean 1 and variance 1/4. The top plot shows the densities (gamma in green, lognormal in blue), and the lower one shows the densities of the logs:
(Plotting the log of the density of the logs is also useful. That is, taking a log-scale on the y-axis above)
This difference implies that the gamma has more of a tail on the left, and less of a tail on the right; the far right tail of the lognormal is heavier and its left tail lighter. And indeed, if you look at the skewness, of the lognormal and gamma, for a given coefficient of variation, the lognormal is more right skew ($\text{CV}^3+3\text{CV}$) than the gamma ($2\text{CV}$).
-
1$\begingroup$ +1. Do you know if there is a closed formula for the skewness of the log of gamma? For lognormal, the skewness of log is clearly zero, and I am wondering if there is some expression for the gamma. Wikipedia gives formulas for the mean and the variance of log(gamma), but not for the skewness. $\endgroup$– amoebaCommented Nov 8, 2017 at 15:43
-
$\begingroup$ Gradshteyn & Ryzhik (sect 4.358, 7th ed) list explicit closed forms for $\int_0^\infty x^{\nu-1}e^{-\mu x}(\ln x)^p dx$ for $p=2,3,4$ while the $p=1$ case is done in 4.352 (assuming you regard expressions in $\Gamma, \psi$ and $\zeta$ functions as closed form) -- from which it is definitely doable up to kurtosis; they give the integral for all $p$ as a derivative of a gamma function so presumably it's feasible to go higher. So skewness is certainly doable but not especially "neat". If you want to pursue it I could give you the integrals. $\endgroup$– Glen_bCommented Nov 9, 2017 at 2:16
-
$\begingroup$ However we don't need to evaluate the skewness in order to discern its sign. Examining the log of the density of the logs should suffice to establish that because one side clearly dominates the other. $\endgroup$– Glen_bCommented Nov 9, 2017 at 4:03
-
1$\begingroup$ Thanks Glen. I decided to post it as a new question: stats.stackexchange.com/questions/312803. I spent some time searching for a ready answer but could not find any, so it might be valuable for the future to have it written down somewhere where it's easy to find. It might be a somewhat better fit for Math.SE, but I'd prefer to have it here, to be honest. $\endgroup$– amoebaCommented Nov 9, 2017 at 10:11
$\begingroup$
$\endgroup$
1
Yes, the gamma distribution is the maximum entropy distribution for which the mean $E(X)$ and mean-log $E(\log X)$ are fixed. As with all exponential family distributions, it is the unique maximum entropy distribution for a fixed expected sufficient statistic.
To answer your question about physical processes that generate these distributions: The lognormal distribution arises when the logarithm of X is normally distributed, for example, if X is the product of very many small factors. If X is gamma distributed, it is the sum of many exponentially-distributed variates. For example, the waiting time for many events of a Poisson process.
-
7$\begingroup$ No need of "many" exponential variates to be Gamma. $\endgroup$ Commented Oct 10, 2013 at 9:40