1
$\begingroup$

So I would like to derive the score function for my GLM, which in this case happens to be logistic regression. My approach is to write the pdf as an exponential family, I get it from these slides.

I can write the pdf as, where $\theta=\log{\frac{p}{1-p}}$, $b(\theta)=-n\log{(1-p)}$, $a(\phi)=1$ and $c(y,\phi)=\log{\binom{n}{p}}$.

$$\frac{y\theta-b(\theta)}{a(\theta)}+c(y,\theta)$$

As far as I can see I should get, when I differentiate it with regards to $\theta$:

$$\frac{y-\theta}{1}$$

But I cannot really get that result, I instead get $b'(\theta)=\frac{n}{1-p}$, so is my expectations wrong or am I differentiating something wrong? And what would the score function of a logistic regression look like?

EDIT

So from these notes. I get that the score function of a GLM with the following link function $g(\mu)=\log{\frac{\mu}{1-\mu}}$, for the $i$th observation will be:

$$ \frac{y_i-\mu_i}{a(\phi_i)}\frac{1}{b^{\prime\prime}(\theta_i)}\frac{x_i}{g'(\mu_i)} $$

And since $$g'(\mu)=\frac{\mu-1}{\mu(1-\mu)}$$
And $$b^{\prime\prime}(\theta)=\frac{ne^{\eta}}{(e^{\eta}+1)^2}$$

Then I would get that the the score function for a logistic regregssion GLM is:

$$\frac{y_i-\mu_i}{\frac{ne^{\eta_i}}{(e^{\eta_i}+1)^2}}\frac{x_i}{\frac{\mu_i-1}{\mu_i(1-\mu_i)}}$$

So are my inferences correct? And can this statement be made prettier?

As $g'(\mu)=1$ and $b^{\prime\prime}(\theta)=1$ (since $b'(\theta)=\frac{mu^2}{2}$ and $\theta=\mu$) for a GLM where the link function is just $g(\mu)=\mu$, so this formula also holds for getting the score function of a LM right?

$\endgroup$
  • $\begingroup$ I hope this is not bad practice, but I have updated my question and I would like an answer to my updated part, if anyone would be so kind? Thanks in advance :) $\endgroup$ – lo2 Dec 28 '18 at 10:49
  • $\begingroup$ FYI: Link to referenced slides (probably) changed to: analyticable.com/wp-content/uploads/2015/12/… $\endgroup$ – dohmatob Mar 25 at 17:43
1
$\begingroup$

Ok so the main thing is $\theta=\ln\frac{p}{1-p}$, but $b(\theta)$ isn't actually a function of $\theta$ yet! so you need to rearrange for $\theta$ in terms of $p$. First take natural logs: $$e^\theta=\frac{p}{1-p}$$ Therefore, $$p=\frac{e^\theta}{1+e^\theta}$$ Now sub that into $b(\theta)$, so $$b(\theta)=-n(ln(1-\frac{e^\theta}{1+e^\theta}))$$ So now $b'(\theta)$ is $$\frac{ne^\theta}{e^\theta+1}$$ Now sub the $p=\frac{e^\theta}{e^\theta+1}$ back in $$np$$ which is the mean of the binomial distribution.

$\endgroup$
  • $\begingroup$ I just edited a minus to a plus, i typoed it by mistake. $\endgroup$ – Huy Pham Dec 9 '18 at 4:04
  • $\begingroup$ Thanks a lot :) I have just added a bit to my original question! $\endgroup$ – lo2 Dec 12 '18 at 20:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.