In Generalized Linear Models by Peter McCullagh, John Ashworth Nelder,

  1. the exponential family distribution has the following form: $$ f_Y(y; \theta, \phi) = \exp \{ (y\theta - b(\theta))/a(\phi) + c(y, \phi)\} (2.4) $$ Then the mean is $$ EY = b'(\theta) $$

    I was wondering if in general, is $b'$ assumed to be invertible or injective in generalized linear models, so that we can write $\theta = b'^{-1}(EY)$?

  2. The mean is related to the linear predictor via $g(EY) = X \beta$. Is the link function $g$ assumed to be injective or invertible in generalized linear models, so that $EY= g^{-1}(X\beta)$?

The purpose of the two questions is to help me to understand how the parameter $\beta$ in generalized linear models is estimated via MLE. In particular, can the pdf of $Y$ be written as $f_Y(y; \beta, X, \phi)$, for example, by $\theta = b'^{-1}(g^{-1}(X\beta))$, so that we can apply MLE on $f_Y(y; \beta, X, \phi)$? Thanks!


For the title, yes, any likelihood function is in terms of parameters and input variables.

The answers to your next two questions are generally "No". You can take a look at Table 2.1 on page 30 of the book McCullagh and Nelder (1989). There are two rows for $b(\theta)$ and link function of common distributions. For example, $b(\theta)=\theta^2/2$ for normal distribution and $b(\theta)=-(-2\theta)^{1/2}$ for inverse Gaussian. The canonical link function for inverse Gaussian is $1/\mu^2$.

In this way, parameters can be estimated via MLE with derivatives of likelihood function like score function, not as easy as your formula. I believe you can find the answer in the book.

| cite | improve this answer | |
  • $\begingroup$ Thanks! Do you mean the likelihood function may not be able to be rewritten in terms of $\beta$ and $X$? How does MLE estimate $\beta$ then? (or if the book has the answer, where is it?) $\endgroup$ – Tim Aug 19 '13 at 0:47
  • $\begingroup$ As I said, "any likelihood function is in terms of parameters and input variables." You can find parameter estimation in the book, for example, Section 4.4 staring from page 114 for binary data. $\endgroup$ – Randel Aug 19 '13 at 1:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.