Fisher information matrix for logistic regression using the logit link

Question

A question on a problem sheet I am working on asks us to find the Fisher information matrix (FIM) for the following setup:
Suppose that $Y_i \sim \textrm{Bin}(r_i, \pi_i)$ for $i = 1, 2, \dots , n$, all independent, where the $r_i$ are known, $\ln({\pi_i/(1 − \pi_i)}) = \beta_0 + \beta_1 x_i$ and $x_i$ is a known covariate. We would like to find the FIM.
When finding the FIM for a model using the identity link, it is as simple as finding derivatives of the log-likelihood, here it seems more challenging. At first I wrote $\pi_i =\frac{\exp(\beta_0 + \beta_1 x_i)}{1+\exp(\beta_0 + \beta_1 x_i)}$ and attempted to differentiate the log-likelihood with respect to the parameters, but that seems to be incorrect. The solution sheet says that the following solves the problem:
"We know that $\mu_i = \pi_i r_i$ and $\eta_i=\ln(\pi_i/(1 − \pi_i)) =\ln(\mu_i /(r_i - \mu_i))$ So that $$\frac{d \eta_i}{d \mu_i}= \frac{r_i}{\mu_i(r_i- \mu_i)} = \frac{1}{r_i \pi_i(1-\pi_i)} $$Since $\operatorname{Var}(Y_i)=r_i \pi_i(1-\pi_i)$ the Fisher information matrix is given by:

$$V= \begin{pmatrix}\sum_{i=1}^nr_i \pi_i(1-\pi_i) & \sum_{i=1}^nx_ir_i \pi_i(1-\pi_i)\\ \sum_{i=1}^nx_ir_i \pi_i(1-\pi_i) & \sum_{i=1}^nx_i^2r_i \pi_i(1-\pi_i) \end{pmatrix}.$$

I am very unsure how this solution was achieved; can anyone break this down a bit more?

Answer 1

The model setup is that a binomial generalized linear model with logit link, also called logistic regression. There are standard and quite simple formulas for the Fisher information matrix (FIM) of a generalized linear model. You do not need to differentiate the log-likelihood from the scratch because the log-likelihood for a GLM has a special form that simplifies the calculations considerably. Indeed you only need to know the variance var$(y)$ and the link-derivative $\partial\mu/\partial \eta$ mentioned in the solution sheet, then the FIM can be written down immediately. The only difference between the logit link and the identity link is that the $\partial\mu/\partial \eta$ term changes.

You are working through a problem sheet, which is presumably associated with a lecture series or online course on generalized linear models and statistical inference. The course you are studying will presumably have derived formula for the FIM of a generalized linear model, and the purpose of the problem sheet is to prompt you to use the formulas and methods that have been derived in the course. So my advice is to go back to the course that the problem sheet is from and familiarize yourself with the methods that have been presented in the course. Otherwise, working through the problem sheet won't have any learning value for you.

If you want to consult a textbook, we derive the same information matrix as in your question in Equation 4.16 of my book with Peter Dunn (Dunn & Smyth, 2018). We go through the calculations in quite a lot of detail. Our example assumes $r_i=1$ but that makes little difference to the calculations.

Reference

Dunn PK, Smyth GK (2018). Generalized linear models with examples in R. Springer, New York, NY. DOI: 10.1007/978-1-4419-0118-7. ISBN: 978-1-4419-0118-7.