59

The gamma has a property shared by the lognormal; namely that when the shape parameter is held constant while the scale parameter is varied (as is usually done when using either for models), the variance is proportional to mean-squared (constant coefficient of variation). Something approximate to this occurs fairly often with financial data, or indeed, with ...


47

The (right) tail of a distribution describes its behavior at large values. The correct object to study is not its density--which in many practical cases does not exist--but rather its distribution function $F$. More specifically, because $F$ must rise asymptotically to $1$ for large arguments $x$ (by the Law of Total Probability), we are interested in how ...


41

First, combine any sums having the same scale factor: a $\Gamma(n, \beta)$ plus a $\Gamma(m,\beta)$ variate form a $\Gamma(n+m,\beta)$ variate. Next, observe that the characteristic function (cf) of $\Gamma(n, \beta)$ is $(1-i \beta t)^{-n}$, whence the cf of a sum of these distributions is the product $$\prod_{j} \frac{1}{(1-i \beta_j t)^{n_j}}.$$ When ...


35

Jacobians--the absolute determinants of the change of variable function--appear formidable and can be complicated. Nevertheless, they are an essential and unavoidable part of the calculation of a multivariate change of variable. It would seem there's nothing for it but to write down a $k+1$ by $k+1$ matrix of derivatives and do the calculation. There's a ...


33

As for qualitative differences, the lognormal and gamma are, as you say, quite similar. Indeed, in practice they're often used to model the same phenomena (some people will use a gamma where others use a lognormal). They are both, for example, constant-coefficient-of-variation models (the CV for the lognormal is $\sqrt{e^{\sigma^2} -1}$, for the gamma it's ...


31

The gamma and the lognormal are both right skew, constant-coefficient-of-variation distributions on $(0,\infty)$, and they're often the basis of "competing" models for particular kinds of phenomena. There are various ways to define the heaviness of a tail, but in this case I think all the usual ones show that the lognormal is heavier. (What the first person ...


29

When you're considering simple parametric models for the conditional distribution of data (i.e. the distribution of each group, or the expected distribution for each combination of predictor variables), and you are dealing with a positive continuous distribution, the two common choices are Gamma and log-Normal. Besides satisfying the specification of the ...


28

That's a good question. In fact, why don't people use generalised linear models (GLM) more is also a good question. Warning note: Some people use GLM for general linear model, not what is in mind here. It does depend where you look. For example, gamma distributions have been popular in several of the environmental sciences for some decades and so ...


26

Both the gamma and Weibull distributions can be seen as generalisations of the exponential distribution. If we look at the exponential distribution as describing the waiting time of a Poisson process (the time we have to wait until an event happens, if that event is equally likely to occur in any time interval), then the $\Gamma(k, \theta)$ distribution ...


25

The log-linked gamma GLM specification is identical to exponential regression: $$E[y \vert x,z] = \exp \left( \alpha + \beta \cdot x +\gamma \cdot z \right)=\hat y$$ This means that $E[y \vert x=0,z=0]=\exp(\alpha)$. That's not a very meaningful value (unless you centered your variables to be be mean zero beforehand). There are at least three way to ...


24

Good effort for thinking through this issue. Here's an incomplete answer, but some starters for the next steps. First, the AIC scores - based on likelihoods - are on different scales because of the different distributions and link functions, so aren't comparable. Your sum of squares and mean sum of squares have been calculated on the original scale and ...


24

An alternative is the approach of Kooperberg and colleagues, based on estimating the density using splines to approximate the log-density of the data. I'll show an example using the data from @whuber's answer, which will allow for a comparison of approaches. set.seed(17) x <- rexp(1000) You'll need the logspline package installed for this; install it if ...


23

Wikipedia has a page that lists many probability distributions with links to more detail about each distribution. You can look through the list and follow the links to get a better feel for the types of applications that the different distributions are commonly used for. Just remember that these distributions are used to model reality and as Box said: "all ...


22

One solution, borrowed from approaches to edge-weighting of spatial statistics, is to truncate the density on the left at zero but to up-weight the data that are closest to zero. The idea is that each value $x$ is "spread" into a kernel of unit total area centered at $x$; any part of the kernel that would spill over into negative territory is removed and ...


19

I will outline how the problem can be approached and state what I think the end result will be for the special case when the shape parameters are integers, but not fill in the details. First, note that $X-Y$ takes on values in $(-\infty,\infty)$ and so $f_{X-Y}(z)$ has support $(-\infty,\infty)$. Second, from the standard results that the density of the ...


19

Well, quite clearly the log-linear fit to the Gaussian is unsuitable; there's strong heteroskedasticity in the residuals. So let's take that out of consideration. What's left is lognormal vs gamma. Note that the histogram of $T$ is of no direct use, since the marginal distribution will be a mixture of variates (each conditioned on a different set of values ...


18

As Prof. Sarwate's comment noted, the relations between squared normal and chi-square are a very widely disseminated fact - as it should be also the fact that a chi-square is just a special case of the Gamma distribution: $$X \sim N(0,\sigma^2) \Rightarrow X^2/\sigma^2 \sim \mathcal \chi^2_1 \Rightarrow X^2 \sim \sigma^2\mathcal \chi^2_1= \text{Gamma}\left(\...


17

The sum of $n$ independent Gamma random variables $\sim \Gamma(t_i, \lambda)$ is a Gamma random variable $\sim \Gamma\left(\sum_i t_i, \lambda\right)$. It does not matter what the second parameter means (scale or inverse of scale) as long as all $n$ random variable have the same second parameter. This idea extends readily to $\chi^2$ random variables which ...


17

In a more general way, $E[\ln(Y|x)]$ and $\ln([E(Y|X])$ are not the same. Also the variance assumptions made by GLM are more flexible than in OLS, and for certain modeling situation as counts variance can be different taking distinct distribution families. About the distribution family in my opinion is a question about the variance and its relation with the ...


17

Some background The $\chi^2_n$ distribution is defined as the distribution that results from summing the squares of $n$ independent random variables $\mathcal{N}(0,1)$, so: $$\text{If }X_1,\ldots,X_n\sim\mathcal{N}(0,1)\text{ and are independent, then }Y_1=\sum_{i=1}^nX_i^2\sim \chi^2_n,$$ where $X\sim Y$ denotes that the random variables $X$ and $Y$ have ...


17

This one (maybe surprisingly) can be done with easy elementary operations (employing Richard Feynman's favorite trick of differentiating under the integral sign with respect to a parameter). We are supposing $X$ has a $\Gamma(\alpha,\beta)$ distribution and we wish to find the expectation of $Y=\log(X).$ First, because $\beta$ is a scale parameter, its ...


16

The proof is as follows: (1) Remember that the characteristic function of the sum of independent random variables is the product of their individual characteristic functions; (2) Get the characteristic function of a gamma random variable here; (3) Do the simple algebra. To get some intuition beyond this algebraic argument, check whuber's comment. Note: The ...


16

$\beta_1X \sim Gamma(\alpha_1, 1)$ and $\beta_2 Y \sim Gamma(\alpha_2, 1)$, then according to Wikipedia $$\dfrac{\beta_1X}{\beta_2Y} \sim \text{Beta Prime distribution}(\alpha_1, \alpha_2). $$ In addition, short hand you write $\beta'(\alpha_1, \alpha_2)$. Now the Wiki page also describes the density of the general Beta-prime distribution $\beta'(\alpha_1,...


15

Here is an answer that does not need to use characteristic functions, but instead reinforces some ideas that have other uses in statistics. The density of the sum of independent random variables is the convolutions of the densities. So, taking $\theta = 1$ for ease of exposition, we have for $z > 0$, $$\begin{align} f_{X+Y}(z) &= \int_0^z f_X(x)f_Y(z-...


15

Both the MLEs and moment based estimators are consistent and so you'd expect that in sufficiently large samples from a gamma distribution they'd tend to be quite similar. However, they won't necessarily be alike when the distribution is not close to a gamma. Looking at the distribution of the log of the data, it is roughly symmetric - or indeed actually ...


15

The answer by @whuber is quite nice; I will essentially restate his answer in a more general form which connects (in my opinion) better with statistical theory, and which makes clear the power of the overall technique. Consider a family of distributions $\{F_\theta : \theta \in \Theta\}$ which consitute an exponential family, meaning they admit a density $...


14

This is a fairly straight forward problem. Although there is a connection between the Poisson and Negative Binomial distributions, I actually think this is unhelpful for your specific question as it encourages people to think of negative binomial processes. Basically, you have a series of Poisson processes: $$Y_i(t_i)|\lambda_i\sim Poisson(\lambda_i t_i)$$...


14

You can use the free statistical program R, available at http://www.r-project.org/, to analyze your data. You state that you want to determine how your data are distributed. You guess that your data either conform to a beta or a gamma distribution. You provide no information about your data and do not speculate that it could conform to one of the many other ...


14

I have some tips : (1) How residuals ought to compare to fits isn't always all that obvious, so it's good to be familiar with diagnostics for particular models. In logistic regression models, for example, the Hosmer-Lemeshow statistic is used to assess goodness of fit; leverage values tend to be small where the estimated odds are very large, very small or ...


14

I used the gamma.shape function of MASS package as described by Balajari (2013) in order to estimate the shape parameter afterwards and then adjust coefficients estimations and predictions in the GLM. I advised you to read the lecture as it is, in my opinion, very clear and interesting concerning the use of gamma distribution in GLMs. glmGamma <- glm(...


Only top voted, non community-wiki answers of a minimum length are eligible