# GLM with mean dependent Variance

Assume we wanted to perform a linear regression, but we assume that the standard deviation is proportional to the mean, i.e.

$$y(x) \sim \mathcal N(\mu(x), c\mu^2(x))$$

where $$c$$ is a known constant. I tried to write down the glm of $$y$$ in standard form

$$f_{Y}(y | \theta, \tau)=h(y, \tau) \exp \left(\frac{b(\theta) T(y)-A(\theta)}{d(\tau)}\right)$$

however expanding the exponential yields:

\begin{align} \frac{1}{\sqrt{2\pi c\mu^2}} e^{-\frac{1}{2}\frac{(y-\mu)^2}{c\mu^2}} &=\exp\Big(-\frac{1}{2}\big(\frac{y^2}{c\mu^2} - 2\frac{y\mu}{c\mu^2}+\frac{\mu^2}{c\mu^2} \big) - \frac{1}{2}\log(2\pi c \mu^2)) \\ &=\exp\Big(\frac{-\frac{1}{2}y^2}{c\mu^2} + \frac{y}{c\mu} -\log \mu - \frac{1}{2c}- \frac{1}{2}\log(2\pi c)\Big) \\ \end{align}

And here it seems that we are almost done, as we can identify

\begin{align} \tau = d(\tau) = c, \quad h(y,\tau) = \exp(- \frac{1}{2c}- \frac{1}{2}\log(2\pi c)) \\ b(\theta) = \begin{pmatrix}1/\mu^2\\1/\mu \end{pmatrix},\qquad T(y) = \begin{pmatrix} -\frac{1}{2}y^2 \\y \end{pmatrix}, \qquad A(\theta) = \log(\mu) \end{align}

However, it doesn't add quite up: it would have to be $$-\frac{\log\mu}{c}$$ above, otherwise we are missing this term.

Does this mean that this is not a valid glm model?

• It is certainly not a valid GLM model. The only GLM family that has variance proportion to the mean-squared is the gamma family. Commented Apr 17, 2019 at 5:12

As confirmed by @gordon-smyth, it is not a valid GLM model for the reason explained in the question. That is the set of all distributions $$N(\mu, c \mu^2)$$ for $$\mu\in \mathbb R, c\in \mathbb R_+$$ does not constitute an exponential dispersion model .
You may be interested in looking at double generalized linear models, which allows the very similar model $$N(X\beta_1, (X\beta_2)^2)$$. Another alternative option (again pointed out by @gordon-smyth) is to use the gamma distribution which satisfies that
$$\Gamma(\text{mean}=\mu, \text{variance}=c\mu^2)$$ is an exponential dispersion model for $$\mu>0, c>0$$.
• Hm. The issue with a Gamma is that the mean is restricted to $(0, \infty)$. I guess what still confuses me about GLMs is what are the free choices I have when setting up the model. Some authors introduce a variance function $Var(Y)=\phi V(\mu)$ and make it sound like it is a free choice. So is it the case that with GLM we can not freely choose all 3 of {exp-family distribution, link-function, variance-function}, but only two of them? Commented Apr 17, 2019 at 15:25
• Ah ok. Actually I think I also found another way to see it. If mean=$\mu$ and var=$c\mu^2$, then $A'(\theta)=\mu$ and $A''(\theta)=c\mu^2$, implying $\mu'(\theta) = c\mu^2(\theta)$. This ODE has the solution $\mu(\theta) = 1/(a-c\theta)$, so in the special case $a=0, c=1$ we have $\mu(\theta)=-1/\theta$. But this means that the link function must be the negative inverse! So choosing the variance function already fixes the link function as they are connected by a differential equation through the cumulant. Commented Apr 17, 2019 at 16:18
• It is true that $\mu(\theta)=-1/\theta$ in the gamma glm, but that does not mean that the link function must be $-1/x$. The link function, $g$ connects the linear predictor $\eta=X\beta$ to $\mu$. That is $g(\eta)=\mu$. If, in this gamma case, $g(x)=-1/x$, we would have $g^{-1}(\mu)=\theta=\eta$, and then we would call $g$ the canonical link. Commented Apr 17, 2019 at 21:20