4
$\begingroup$

The Beta distribution is: $$p(y)=\frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)}y^{\alpha-1}(1-y)^{\beta-1} $$

It's part of the exponential family.

We can reparametrize this with using mean and dispersion $\mu=\frac{\alpha}{\alpha+\beta}, \phi = \alpha+\beta$ and get:

$$ \mu=\frac{\alpha}{\alpha+\beta}, \phi = \alpha+\beta \\ \Rightarrow \mu = \frac{\alpha}{\phi} \Rightarrow \alpha = \mu\phi \\ \beta = \phi-\alpha=\phi-\mu\phi=\phi(1-\mu) \\ p(y)=\frac{\Gamma(\phi)}{\Gamma(\mu\phi)\Gamma(\phi(1-\mu))}y^{\mu\phi-1}(1-y)^{\phi(1-\mu)-1} \\ = \frac{\Gamma(\phi)}{y(1-y)^{\phi-1}}\exp\{\mu\phi\log y-\phi\mu\log (1-y)-[\log \Gamma(\mu\phi)+\log\Gamma(\phi(1-\mu))]\} \\ = \frac{\Gamma(\phi)}{y(1-y)^{\phi-1}}\exp\{\phi(\mu\log \frac{y}{1-y}-\frac{1}{\phi}[\log \Gamma(\mu\phi)+\log\Gamma(\phi(1-\mu))])\} $$

This suggests that it's part of the (GLM) exponential family, with

$$ a(\phi) = \frac{1}{\phi} \\ t(y) = \log \frac{y}{1-y} \\ \theta = \mu \\ b(\theta) =\frac{1}{\phi}[\log \Gamma(\mu\phi)+\log\Gamma(\phi(1-\mu))] \\ h(y,\phi) = \frac{\Gamma(\phi)}{y(1-y)^{\phi-1}} $$

There are a few problems with this:

  1. The log-normalizer $b(\theta)$ seems to be dependent on the dispersion parameter (which usually doesn't happen for other distributions)

  2. The mean-variance relationship in GLM's satisfy: $\mathbb V[y] = a(\phi)V(\mu)$, yet here we get that $\mathbb V[y]=\frac{1}{\phi+1}\mu(1-\mu)$, that is that $a(\phi) = \frac{1}{\phi+1}$ and not $\frac{1}{\phi}$

  3. Taking the derivative of the log-normalizer w.r.t. the natural parameter doesn't seem to give the mean (we get $\psi(\alpha)-\psi(\beta)$, where $\psi$ is the digamma function, but I don't see how this is equal to $\frac{\alpha}{\alpha+\beta}$)

Here they claim "that the distribution of the response is not a member of the exponential family"...

So who's right? Wikipedia, or the paper guys? And how can we write the Beta distribution in GLM Expo-Family form? Can one parameterization of a distribution be part of the Expo-Family, and another not???

$\endgroup$
1

1 Answer 1

7
$\begingroup$

GLMs assume that the response distribution is an Exponential Dispersion Model (EDM): $$y_i \sim \mbox{ED}(\mu_i,\phi/w_i)$$ where $\phi$ is the dispersion parameter and $w_i$ is a known weight.

EDMs are both more special and more general than exponential families. If the dispersion $\phi$ is known, then an EDM is a single-parameter linear exponential family (LEF). LEFs are a particular simple type of exponential family so, in this sense, EDMs are a subset of exponential families. On the other hand, an EDM with unknown $\phi$ does not need to be a two-parameter exponential family, so EDMs are in this sense more general than exponential families.

An EDM ED$(\mu,\phi)$ density is writeable in the following form: $$f(y;\mu,\phi) = a(y,\phi) \exp(\frac{d(y,\mu)}{-2\phi})$$ where $d(y,\mu)$ is an (asymmetric) distance measure between $y$ and $\mu$. This form for the density clarifies that $\mu$ is a location parameter and $\phi$ is a dispersion parameter.

The Beta distribution is a two-parameter exponential family but not an EDM.

If $\phi=\alpha+\beta$ is known, then the Beta distribution can be transformed to a LEF, which is what you implicitly discovered in your calculations. The logit$(y)$ random variable $$z = \log\left(\frac{y}{1-y}\right)$$ follows a logistic-beta distribution, see Is there a "beta distribution" over the entire real line?. The logistic-beta distribution with known $\phi$ is a LEF, so it could be used to generate a GLM family, although I have never known anyone to do so. Note that this derivation only works when $\alpha+\beta$ is known; $\alpha+\beta$ would not be the dispersion parameter for a logistic-beta GLM.

$\endgroup$
2
  • $\begingroup$ $d(y,\mu)$ must have $y$ linear when it multiplies the natural parameter $\theta \cdot y$, no? $\endgroup$ Commented Sep 8, 2022 at 7:53
  • $\begingroup$ @MaverickMeerkat $d(y,\mu)$ is the unit deviance and it is compatible with the LEF parametrization for the density. We discuss the unit deviance in some detail in my book with Peter Dunn on Generalized Linear Models with Examples in R. $\endgroup$ Commented Sep 11, 2022 at 7:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.