# Fisher information matrix for logistic regression using the logit link

A question on a problem sheet I am working on asks us to find the Fisher information matrix (FIM) for the following setup:
Suppose that $$Y_i \sim \textrm{Bin}(r_i, \pi_i)$$ for $$i = 1, 2, \dots , n$$, all independent, where the $$r_i$$ are known, $$\ln({\pi_i/(1 − \pi_i)}) = \beta_0 + \beta_1 x_i$$ and $$x_i$$ is a known covariate. We would like to find the FIM.
When finding the FIM for a model using the identity link, it is as simple as finding derivatives of the log-likelihood, here it seems more challenging. At first I wrote $$\pi_i =\frac{\exp(\beta_0 + \beta_1 x_i)}{1+\exp(\beta_0 + \beta_1 x_i)}$$ and attempted to differentiate the log-likelihood with respect to the parameters, but that seems to be incorrect. The solution sheet says that the following solves the problem:
"We know that $$\mu_i = \pi_i r_i$$ and $$\eta_i=\ln(\pi_i/(1 − \pi_i)) =\ln(\mu_i /(r_i - \mu_i))$$ So that $$\frac{d \eta_i}{d \mu_i}= \frac{r_i}{\mu_i(r_i- \mu_i)} = \frac{1}{r_i \pi_i(1-\pi_i)}$$Since $$\operatorname{Var}(Y_i)=r_i \pi_i(1-\pi_i)$$ the Fisher information matrix is given by:

$$V= \begin{pmatrix}\sum_{i=1}^nr_i \pi_i(1-\pi_i) & \sum_{i=1}^nx_ir_i \pi_i(1-\pi_i)\\ \sum_{i=1}^nx_ir_i \pi_i(1-\pi_i) & \sum_{i=1}^nx_i^2r_i \pi_i(1-\pi_i) \end{pmatrix}.$$

I am very unsure how this solution was achieved; can anyone break this down a bit more?

• You seem to be confusing information and variance for one point. The asymptotic GLM result is that $\sqrt{n} \left( \hat{\beta} - \beta \right) \rightarrow_d \mathcal{N}(0, \mathcal{I}^{-1}(\beta))$ Commented Dec 9, 2022 at 17:55
• @AdamO The only reference to variance for one point in the question is in the quoted solution from the solution sheet. The whole point of the question is to compute $\cal I$ and the formula for $\cal I$ in standard GLM theory involves the variance function, so the variance of an individual point is relevant. Commented Dec 10, 2022 at 21:05
• @GordonSmyth I don't understand your point. The question clearly has inconsistencies referring to variance ("V") and calling it information. Commented Dec 11, 2022 at 0:22
• @AdamO The V matrix is inside quotes so, if it was wrong, then it would be fault of the solution sheet rather than the fault of OP. However the V matrix shown in the question is in fact the correct information matrix for this logistic regression. The reason why the entries look like variances is because the logit link derivatives in this case are equal to inverse variances. Commented Dec 11, 2022 at 1:02
• @AdamO It is also worth clarifying that $\cal I$ in your comment represents average information per observation, i.e., it is the usual information matrix divided by $n$. The information matrix that is used in the glm fit, and which is being derived by OP, is not divided by $n$. Commented Dec 11, 2022 at 2:46

The model setup is a binomial generalized linear model with logit link, also called logistic regression. There are standard and quite simple formulas for the Fisher information matrix (FIM) of a generalized linear model. You do not need to differentiate the log-likelihood from the scratch because the log-likelihood for a GLM has a special form that simplifies the calculations considerably. Indeed you only need to know the variance var$$(y)$$ and the link-derivative $$\partial\mu/\partial \eta$$ mentioned in the solution sheet, then the FIM can be written down immediately. The only difference between the logit link and the identity link is that the $$\partial\mu/\partial \eta$$ term changes.
If you want to consult a textbook, we derive the same information matrix as in your question in Equation 4.16 of my book with Peter Dunn (Dunn & Smyth, 2018). We go through the calculations in quite a lot of detail. Our example assumes $$r_i=1$$ but that makes little difference to the calculations.