37
$\begingroup$

The premise is this quote from vignette of R package betareg1.

Further-more, the model shares some properties (such as linear predictor, link function, dispersion parameter) with generalized linear models (GLMs; McCullagh and Nelder 1989), but it is not a special case of this framework (not even for fixed dispersion)

This answer also makes allusion to the fact:

[...] This is a type of regression model that is appropriate when the response variable is distributed as Beta. You can think of it as analogous to a generalized linear model. It's exactly what you are looking for [...] (emphasis mine)

Question title says it all: why Beta/Dirichlet Regression are not considered Generalized Linear Models (are they not)?


As far as I know, the Generalized Linear Model defines models built on the expectation of their dependent variables conditional on the independent ones.

$f$ is the link function that maps the expectation, $g$ is probability distribution, $Y$ the outcomes and $X$ the predictiors, $\beta$ are linear parameters and $\sigma^2$ the variance.

$$f\left(\mathbb E\left(Y\mid X\right)\right) \sim g(\beta X, I\sigma^2)$$

Different GLMs impose (or relax) the relationship between the mean and the variance, but $g$ must be a probability distribution in the exponential family, a desirable property which should improve robustness of the estimation if I recall correctly. The Beta and Dirichlet distributions are part of the exponential family, though, so I'm out of ideas.


[1] Cribari-Neto, F., & Zeileis, A. (2009). Beta regression in R.

$\endgroup$
7
  • 2
    $\begingroup$ (+1) Related: stats.stackexchange.com/a/189196. $\endgroup$
    – amoeba
    Sep 22, 2017 at 19:55
  • $\begingroup$ @amoeba Thanks for the link, hadn't seen that question before. $\endgroup$
    – Firebug
    Sep 22, 2017 at 19:58
  • 2
    $\begingroup$ I think the issue is that while if you write the beta distribution with the standard $a$, $b$ parameters (i.e. $a = b = 1$ implies uniform(0,1)), then the beta distribution is in the exponential family, if you write it in terms of $\mu$ (mean) and $\phi$ (dispersion), it is not. But I've never cared that much for whether a distribution is in the exponential family. $\endgroup$
    – Cliff AB
    Sep 22, 2017 at 23:06
  • $\begingroup$ @CliffAB After reading the comments under Tim's answer below it seems the parametrization of the Beta leads to non-orthogonality of the parameters, which seems to be a requirement for McCullagh-Nelder GLMs. $\endgroup$
    – Firebug
    Sep 23, 2017 at 15:18
  • 1
    $\begingroup$ I think this short answer: stats.stackexchange.com/a/18812/28666 is relevant and adds to the answers here (hinting to why GLMs were originally defined with exponential dispersion family). $\endgroup$
    – amoeba
    Nov 9, 2017 at 9:32

3 Answers 3

24
$\begingroup$

Check the original reference:

Ferrari, S., & Cribari-Neto, F. (2004). Beta regression for modelling rates and proportions. Journal of Applied Statistics, 31(7), 799-815.

as the authors note, the parameters of re-parametrized beta distribution are correlated, so

Note that the parameters $\beta$ and $\phi$ are not orthogonal, in contrast to what is verified in the class of generalized linear regression models (McCullagh and Nelder, 1989).

So while the model looks like a GLM and quacks like a GLM, it does not perfectly fit the framework.

$\endgroup$
13
  • 7
    $\begingroup$ +1 but it would be great to have a more detailed answer. I, personally, don't understand the quote (even after opening the linked paper). Why are these parameters not orthogonal in beta regression?.. Why is this required for GLMs?.. Etc. $\endgroup$
    – amoeba
    Sep 22, 2017 at 20:03
  • 3
    $\begingroup$ @amoeba honestly, I'm not the kind of person who can give you detailed answer on that. I was never that much interested in the theory behind GLM's to have deep enough understanding of such subtleties. McCullagh and Nelder mention this requirement, but I'd need to check their book to see why exactly it is important. If someone would give detailed explanation of why this is an issue, I'd consider issing a bounty for such answer. $\endgroup$
    – Tim
    Sep 22, 2017 at 20:49
  • 13
    $\begingroup$ The orthogonality requirement in GLMs is important: It means that you can estimate the equation $g(\mu) = x^\top \beta$ without worrying about misspecifying the rest of the likelihood. Parameter estimates are consistent if the mean equation above is correctly specified. Inference is valid if additionally the variance is correctly specified. However, in beta regression you cannot separate the two model equations in this way, even if $\phi$ is just a constant. For consistent results everything has to be specified correctly. $\endgroup$ Sep 22, 2017 at 23:33
  • 3
    $\begingroup$ @AchimZeileis I remembered that I saw your name on CV. What you say makes perfect sense. Maybe you'd like to transform your comment to answer by adding some more rationale? As I said, I'd be happy to award bounty for someone giving detailed enough answer for the question. $\endgroup$
    – Tim
    Sep 23, 2017 at 6:48
  • 3
    $\begingroup$ @Firebug I think a more detailed answer to this Q will already clarify this issue, so I won't start another question any time soon. I'd rather put a bounty here. $\endgroup$
    – amoeba
    Sep 23, 2017 at 18:29
9
$\begingroup$

The answer by @probabilityislogic is on the right track.

The beta distribution is in the two parameter exponential family. The simple GLM models described by Nelder and Wedderburn (1972) do not include all of the distributions in the two parameter exponential family.

In terms of the article by N&W, the GLM applies to the density functions of the following type (this was later named exponential dispersion family in Jørgensen 1987):

$$\pi(z;\theta,\phi) = \exp \left[ \alpha(\phi) \lbrace z\theta - g(\theta) +h(z)\rbrace +\beta(\phi,z) \right]$$

with an additional link function $f()$ and linear model for the natural parameter $\theta = f(\mu) = f(X\beta)$.


So we could rewrite the above distribution also:

$$\pi(z;\mu,\phi) = exp \left[z(f(\mu)\alpha(\phi)) +h(z)\alpha(\phi) - g(f(\mu))\alpha(\phi) +\beta(\phi,z) \right]$$

The two parameter exponential family is:

$$ f(z;\theta_1,\theta_2) = exp \left[T_1(z)\eta_1(\theta_1,\theta_2) + T_2(z)\eta_2(\theta_1,\theta_2) - g(\theta_1,\theta_2) +h(z) \right] $$

which looks similar but more general (also if one of the $\theta$ is constant).


The difference is clear, and also putting the beta distribution in a form as a GLM is not possible.

However, I lack sufficient understanding to create a more intuitive and well informed answer (I have a feeling that there can be much deeper and more elegant relationships to a variety of fundamental principles). The GLM generalizes the distribution of the error by using a single variate exponential dispersion model in place of a least squares model and generalizes the linear relationship in the mean, by using a link function.

The best and most simple intuition seems to be the dispersion-$\alpha(\phi)$-term in the exponential, which gets multiplied with everything and thus the dispersion does not vary with $\theta$. Whereas several two parameter exponential families, and quasi-likelihood methods, allow the dispersion parameter to be a function of $\theta$ as well.

$\endgroup$
5
  • $\begingroup$ The second parameter $\phi$ in the N&W defined df is the dispersion. It is extends the one parameter natural exponential family $\pi(z;\theta)$ $\endgroup$ Oct 12, 2017 at 18:47
  • 1
    $\begingroup$ @amoeba beta is a bivariate exponential family distribution, e.g. www2.stat.duke.edu/courses/Spring11/sta114/lec/expofam.pdf $\endgroup$
    – Tim
    Oct 12, 2017 at 19:28
  • 2
    $\begingroup$ I am not sure if it is not entirely possible, even with fixed dispersion. At least not according to the glm as stated by N&W (what I do know is that lots of people do much more difficult things to solve beta regression). I will edit the answer to show what happens, and where it goes wrong, if we do try to follow the same path of iterative reweighted least squares. $\endgroup$ Oct 12, 2017 at 20:29
  • 2
    $\begingroup$ I have edited the answer somewhat. 1) My initial description of the families and dispersion models was incorrect. The GLM does include all distributions of the one parameter exponential families because it is not just that density function, but also a link function. 2) In terms of a better intuitive view I could not get far and don't expect to get far soon. The GLM models relate to the classical model in various representations, adding weights to the matrix formulation of fitting procedures, derivatives of the log-likelihood functions including terms with the link function and variance,..... $\endgroup$ Oct 13, 2017 at 4:06
  • 2
    $\begingroup$ I took the liberty to edit your answer a little bit, hope you are fine with the edits. Also, it looks like this answer stats.stackexchange.com/a/18812/28666 hints to why N&W used this particular distribution family and not a broader one. $\endgroup$
    – amoeba
    Nov 9, 2017 at 9:35
3
$\begingroup$

I don't think the beta distribution is part of the exponential dispersion family. To get this, you need to have a density

$$f (y;\theta,\tau)=\exp\left (\frac {y\theta - c (\theta)}{\tau} + d (y,\tau)\right)$$

for specified functions $c ()$ and $d () $. The mean is given as $ c'(\theta)$ and the variance is given as $\tau c''(\theta) $. The parameter $\theta $ is called the canonical parameter.

The beta distribution cannot be written this way - one way to see this is by noting there is no $y $ term in the log likelihood - it has $\log [y] $ and $\log [1-y] $ instead

$$f_{beta}(y;\mu,\phi)=\exp\left (\phi\mu\log\left[\frac {y}{1-y}\right] +\phi\log [1-y] - \log [B (\phi\mu,\phi (1-\mu)]-\log\left[\frac {y}{1-y}\right]\right) $$

Yet another way to see that beta is not exponential dispersion family is that it can be written as $y=\frac {x}{x+z} $ where $x $ and $z $ are independent and both follow gamma distributions with the same scale parameter (and gamma is exponential family).

$\endgroup$
8
  • 2
    $\begingroup$ This answer is not correct as written. One way to see this is that, according to the logic presented, the Bernoulli and binomial distributions, for example, would not be in the class of exponential families either. $\endgroup$
    – cardinal
    Sep 27, 2017 at 1:51
  • 2
    $\begingroup$ Sorry, you are correct that the example I gave was erroneous. (Warning: mental arithmetic and mobile use of CrossValidated can be hazardous!) However, my point still stands. This answer is incorrect because it opts for a very narrowly "defined" concept of "exponential family"---much narrower than any conventional source or practical use. $\endgroup$
    – cardinal
    Sep 27, 2017 at 4:45
  • 2
    $\begingroup$ Hmm. Wikipedia does list beta in the list of exponential family distributions. $\endgroup$
    – amoeba
    Sep 27, 2017 at 7:07
  • 1
    $\begingroup$ True - I was thinking of the natural exponential family - which is a special case $\endgroup$ Sep 27, 2017 at 9:12
  • 1
    $\begingroup$ The parameter $\theta$ in the function is also described by a link function, and then this narrowly defined distribution function becomes more wide, including all the distributions of the one parameter exponential family, but only some of the two parameter exponential family. $\endgroup$ Oct 13, 2017 at 4:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.