1
$\begingroup$

Assume we wanted to perform a linear regression, but we assume that the standard deviation is proportional to the mean, i.e.

$$ y(x) \sim \mathcal N(\mu(x), c\mu^2(x)) $$

where $c$ is a known constant. I tried to write down the glm of $y$ in standard form

$$f_{Y}(y | \theta, \tau)=h(y, \tau) \exp \left(\frac{b(\theta) T(y)-A(\theta)}{d(\tau)}\right)$$

however expanding the exponential yields:

\begin{align} \frac{1}{\sqrt{2\pi c\mu^2}} e^{-\frac{1}{2}\frac{(y-\mu)^2}{c\mu^2}} &=\exp\Big(-\frac{1}{2}\big(\frac{y^2}{c\mu^2} - 2\frac{y\mu}{c\mu^2}+\frac{\mu^2}{c\mu^2} \big) - \frac{1}{2}\log(2\pi c \mu^2)) \\ &=\exp\Big(\frac{-\frac{1}{2}y^2}{c\mu^2} + \frac{y}{c\mu} -\log \mu - \frac{1}{2c}- \frac{1}{2}\log(2\pi c)\Big) \\ \end{align}

And here it seems that we are almost done, as we can identify

\begin{align} \tau = d(\tau) = c, \quad h(y,\tau) = \exp(- \frac{1}{2c}- \frac{1}{2}\log(2\pi c)) \\ b(\theta) = \begin{pmatrix}1/\mu^2\\1/\mu \end{pmatrix},\qquad T(y) = \begin{pmatrix} -\frac{1}{2}y^2 \\y \end{pmatrix}, \qquad A(\theta) = \log(\mu) \end{align}

However, it doesn't add quite up: it would have to be $-\frac{\log\mu}{c}$ above, otherwise we are missing this term.

Does this mean that this is not a valid glm model?

$\endgroup$
1
  • 1
    $\begingroup$ It is certainly not a valid GLM model. The only GLM family that has variance proportion to the mean-squared is the gamma family. $\endgroup$ Commented Apr 17, 2019 at 5:12

1 Answer 1

3
$\begingroup$

As confirmed by @gordon-smyth, it is not a valid GLM model for the reason explained in the question. That is the set of all distributions $N(\mu, c \mu^2)$ for $\mu\in \mathbb R, c\in \mathbb R_+$ does not constitute an exponential dispersion model .

You may be interested in looking at double generalized linear models, which allows the very similar model $N(X\beta_1, (X\beta_2)^2)$. Another alternative option (again pointed out by @gordon-smyth) is to use the gamma distribution which satisfies that

$$ \Gamma(\text{mean}=\mu, \text{variance}=c\mu^2) $$ is an exponential dispersion model for $\mu>0, c>0$.

$\endgroup$
4
  • $\begingroup$ Hm. The issue with a Gamma is that the mean is restricted to $(0, \infty)$. I guess what still confuses me about GLMs is what are the free choices I have when setting up the model. Some authors introduce a variance function $Var(Y)=\phi V(\mu)$ and make it sound like it is a free choice. So is it the case that with GLM we can not freely choose all 3 of {exp-family distribution, link-function, variance-function}, but only two of them? $\endgroup$
    – Hyperplane
    Commented Apr 17, 2019 at 15:25
  • 1
    $\begingroup$ Yes, that is a caveat of using the gamma distribution! In exponential dispersion models, the distribution is equivalent to the variance function, so you can not choose the variance function freely if you have already chosen the distribution. link function can always be chosen freely, though (within limits). $\endgroup$
    – svendvn
    Commented Apr 17, 2019 at 15:49
  • $\begingroup$ Ah ok. Actually I think I also found another way to see it. If mean=$\mu$ and var=$c\mu^2$, then $A'(\theta)=\mu$ and $A''(\theta)=c\mu^2$, implying $\mu'(\theta) = c\mu^2(\theta)$. This ODE has the solution $\mu(\theta) = 1/(a-c\theta)$, so in the special case $a=0, c=1$ we have $\mu(\theta)=-1/\theta$. But this means that the link function must be the negative inverse! So choosing the variance function already fixes the link function as they are connected by a differential equation through the cumulant. $\endgroup$
    – Hyperplane
    Commented Apr 17, 2019 at 16:18
  • $\begingroup$ It is true that $\mu(\theta)=-1/\theta$ in the gamma glm, but that does not mean that the link function must be $-1/x$. The link function, $g$ connects the linear predictor $\eta=X\beta$ to $\mu$. That is $g(\eta)=\mu$. If, in this gamma case, $g(x)=-1/x$, we would have $g^{-1}(\mu)=\theta=\eta$, and then we would call $g$ the canonical link. $\endgroup$
    – svendvn
    Commented Apr 17, 2019 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.