# Deriving score function for logistic regression

So I would like to derive the score function for my GLM, which in this case happens to be logistic regression. My approach is to write the pdf as an exponential family, I get it from these slides.

I can write the pdf as, where $$\theta=\log{\frac{p}{1-p}}$$, $$b(\theta)=-n\log{(1-p)}$$, $$a(\phi)=1$$ and $$c(y,\phi)=\log{\binom{n}{p}}$$.

$$\frac{y\theta-b(\theta)}{a(\theta)}+c(y,\theta)$$

As far as I can see I should get, when I differentiate it with regards to $$\theta$$:

$$\frac{y-\theta}{1}$$

But I cannot really get that result, I instead get $$b'(\theta)=\frac{n}{1-p}$$, so is my expectations wrong or am I differentiating something wrong? And what would the score function of a logistic regression look like?

## EDIT

So from these notes. I get that the score function of a GLM with the following link function $$g(\mu)=\log{\frac{\mu}{1-\mu}}$$, for the $$i$$th observation will be:

$$\frac{y_i-\mu_i}{a(\phi_i)}\frac{1}{b^{\prime\prime}(\theta_i)}\frac{x_i}{g'(\mu_i)}$$

And since $$g'(\mu)=\frac{\mu-1}{\mu(1-\mu)}$$
And $$b^{\prime\prime}(\theta)=\frac{ne^{\eta}}{(e^{\eta}+1)^2}$$

Then I would get that the the score function for a logistic regregssion GLM is:

$$\frac{y_i-\mu_i}{\frac{ne^{\eta_i}}{(e^{\eta_i}+1)^2}}\frac{x_i}{\frac{\mu_i-1}{\mu_i(1-\mu_i)}}$$

So are my inferences correct? And can this statement be made prettier?

As $$g'(\mu)=1$$ and $$b^{\prime\prime}(\theta)=1$$ (since $$b'(\theta)=\frac{mu^2}{2}$$ and $$\theta=\mu$$) for a GLM where the link function is just $$g(\mu)=\mu$$, so this formula also holds for getting the score function of a LM right?

• I hope this is not bad practice, but I have updated my question and I would like an answer to my updated part, if anyone would be so kind? Thanks in advance :) – lo2 Dec 28 '18 at 10:49
• FYI: Link to referenced slides (probably) changed to: analyticable.com/wp-content/uploads/2015/12/… – dohmatob Mar 25 '19 at 17:43

## 1 Answer

Ok so the main thing is $$\theta=\ln\frac{p}{1-p}$$, but $$b(\theta)$$ isn't actually a function of $$\theta$$ yet! so you need to rearrange for $$\theta$$ in terms of $$p$$. First take natural logs: $$e^\theta=\frac{p}{1-p}$$ Therefore, $$p=\frac{e^\theta}{1+e^\theta}$$ Now sub that into $$b(\theta)$$, so $$b(\theta)=-n(ln(1-\frac{e^\theta}{1+e^\theta}))$$ So now $$b'(\theta)$$ is $$\frac{ne^\theta}{e^\theta+1}$$ Now sub the $$p=\frac{e^\theta}{e^\theta+1}$$ back in $$np$$ which is the mean of the binomial distribution.

• I just edited a minus to a plus, i typoed it by mistake. – Huy Pham Dec 9 '18 at 4:04
• Thanks a lot :) I have just added a bit to my original question! – lo2 Dec 12 '18 at 20:04