So I would like to derive the score function for my GLM, which in this case happens to be logistic regression. My approach is to write the pdf as an exponential family, I get it from these slides.

I can write the pdf as, where $\theta=\log{\frac{p}{1-p}}$, $b(\theta)=-n\log{(1-p)}$, $a(\phi)=1$ and $c(y,\phi)=\log{\binom{n}{p}}$.


As far as I can see I should get, when I differentiate it with regards to $\theta$:


But I cannot really get that result, I instead get $b'(\theta)=\frac{n}{1-p}$, so is my expectations wrong or am I differentiating something wrong? And what would the score function of a logistic regression look like?


So from these notes. I get that the score function of a GLM with the following link function $g(\mu)=\log{\frac{\mu}{1-\mu}}$, for the $i$th observation will be:

$$ \frac{y_i-\mu_i}{a(\phi_i)}\frac{1}{b^{\prime\prime}(\theta_i)}\frac{x_i}{g'(\mu_i)} $$

And since $$g'(\mu)=\frac{\mu-1}{\mu(1-\mu)}$$
And $$b^{\prime\prime}(\theta)=\frac{ne^{\eta}}{(e^{\eta}+1)^2}$$

Then I would get that the the score function for a logistic regregssion GLM is:


So are my inferences correct? And can this statement be made prettier?

As $g'(\mu)=1$ and $b^{\prime\prime}(\theta)=1$ (since $b'(\theta)=\frac{mu^2}{2}$ and $\theta=\mu$) for a GLM where the link function is just $g(\mu)=\mu$, so this formula also holds for getting the score function of a LM right?

  • $\begingroup$ I hope this is not bad practice, but I have updated my question and I would like an answer to my updated part, if anyone would be so kind? Thanks in advance :) $\endgroup$
    – lo2
    Dec 28, 2018 at 10:49
  • $\begingroup$ FYI: Link to referenced slides (probably) changed to: analyticable.com/wp-content/uploads/2015/12/… $\endgroup$
    – dohmatob
    Mar 25, 2019 at 17:43

1 Answer 1


Ok so the main thing is $\theta=\ln\frac{p}{1-p}$, but $b(\theta)$ isn't actually a function of $\theta$ yet! so you need to rearrange for $\theta$ in terms of $p$. First take natural logs: $$e^\theta=\frac{p}{1-p}$$ Therefore, $$p=\frac{e^\theta}{1+e^\theta}$$ Now sub that into $b(\theta)$, so $$b(\theta)=-n(ln(1-\frac{e^\theta}{1+e^\theta}))$$ So now $b'(\theta)$ is $$\frac{ne^\theta}{e^\theta+1}$$ Now sub the $p=\frac{e^\theta}{e^\theta+1}$ back in $$np$$ which is the mean of the binomial distribution.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.