2
$\begingroup$

I came across the following answer to a problem, and I couldn't reconcile the answer with what I found. I'm sure I did something wrong, but I'm not sure where my mistake is.

The model is of the following form:

$E[Y_i] = \mu_i = (\beta_0 + \beta_1x_i)^2$

with the $Y_i$ iid with a Pareto distribution.

The question is whether this constitutes a GLM. My idea was to take the square root link function:

$g(\mu_i) = ((\beta_0 + \beta_1x_i)^2)^{1/2} = \beta_0 + \beta_1x_i = x_i^T\beta$

where $x_i^T = [1,\: x_i]$ and $\beta^T = [\beta_0,\: \beta_1]$.

Since the $g$ is monotone and differentiable, the Pareto distribution is exponential, and $g(\mu_i)$ is linear in the $\beta_i$ terms, this seems to be a valid GLM to me. The solution to the exercise though says that this model "does not have the canonical form, so this is not a GLM." Does anyone see any mistake I could have made? Perhaps I did not understand what "canonical" means here.

Thank you in advance!

EDIT: It turns out I misread the solution. The original text is "The Pareto distribution belongs to the exponential family, but it does not have the canonical form so this is not a GLM." This is in line with @Glen_b -Reinstate Monica's comment

$\endgroup$
2
  • 3
    $\begingroup$ While a Pareto is exponential family, it's not in the exponential dispersion family used in generalized linear models. You can transform the response to be a glm though. $\endgroup$
    – Glen_b
    Nov 23 '19 at 13:35
  • $\begingroup$ @Glen_b-ReinstateMonica - see my edit. It looks like you're right. Could you comment further on what "exponential dispersion family" means (I assume it's tied to the so-called dispersion parameter), and why we only want exponential families with this characteristic? I have a few books on the subject that I'm perusing, but I'm having trouble finding a good answer to this $\endgroup$
    – CLL
    Nov 23 '19 at 13:56
2
$\begingroup$

While a Pareto is exponential family, it's not in the exponential dispersion family used in generalized linear models.

The exponential dispersion model (EDM) extends the natural exponential family by adding a scale parameter that can be interpreted as a dispersion parameter. (This is not needed in all glms but arises in many of them.)

Note that in the exponential family you have a term in the exponent like $\eta(\theta)\cdot T(y)$. The natural exponential family has $\theta\, y$ there (while the EDM has something like $\lambda\theta\, y$). Note for example that the $T$ is an issue; that transformation is how the data enters the model, and that means you'll end up with a model for $E(T(y))$, not $E(y)$.

In some situations this is hardly a problem - e.g. if you can get MLE's for the parameters of $T(y)$ you can typically get MLEs for the parameters of $y$ and so (if required) estimate $E(y)$ from that.

That is, you can sometimes transform the response to be a glm and work with that.

As an example, consider the lognormal, which is exponential family, but not in the extension of the natural exponential family used in GLMs. Nevertheless we can take logs (indeed, if you check, you will see that $T(y)$ is $\log(y)$) and get a normal which in this case is in the required family, from which we can estimate $\mu$ (without regard to $\sigma$) and then obtain a suitable ML estimate of $\sigma$ and hence of the parameters of the lognormal.

This notion extends the value of the GLM outside the exponential dispersion family (but not to the whole of the exponential family). I've used it (for example) to get MLEs of parameters in an inverse gamma model with log-link (by inverting the data and fitting a gamma glm which gives the MLE for the conditional mean, then using an R function to get a MLE of the shape parameter given that glm, then taking it all back to the inverse gamma scale, allowing me to get MLEs of the mean of that original variable).


On the link function, you have to be careful, since $\sqrt{x^2}=|x|\neq x$. Note in particular that if $\beta_0+\beta_1 x$ can go negative, your square root function is going to give you the wrong sign (it will be the positive root), which may lead to incorrect parameter estimates. If the linear predictor is always going to be clearly positive this may not be an issue.

$\endgroup$
2
  • $\begingroup$ Thank you for the in-depth response! You left me with more questions/things I didn't know before which I will have to explore. Thanks again @Glen_b $\endgroup$
    – CLL
    Nov 25 '19 at 21:16
  • 1
    $\begingroup$ I should say something about your link function as well. ... now done $\endgroup$
    – Glen_b
    Nov 25 '19 at 22:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.