1
$\begingroup$

MLE of Logistic regression

Hi all, I am doing a self study of logistic regression and working through the proof of the MLE; please see picture from Montgomery and Peck text. Could someone please fill in the intermediate steps that take us from the right hand side of 14.7 to the final statement in the photo. I understand going from L() to L-L() and the change from the product operator to the summation operator.. but I don't understand the algebraic manipulation to get $\ln(\frac{\pi}{1-\pi})$ on the left hand side or where $1-y_i$ went when it was brought down from the exponent position. I hope that makes sense and I appreciate your help.

$\endgroup$
3
$\begingroup$

$\ln\prod_{i=1}^n \pi_i^{y_i}(1-\pi_i)^{1-y_i}=\sum_{i=1}^n \ln \big (\pi_i^{y_i}.(1-\pi_i)^{1-y_i}\big)=\sum_{i=1}^n \big ( \ln \pi_i^{y_i}+ \ln (1-\pi_i)^{1-y_i}\big)=\sum_{i=1}^n \big (y_i\ln \pi_i +(1-y_i)\ln (1-\pi_i)\big)=\sum_{i=1}^n \big (y_i\ln \pi_i -y_i \ln (1-\pi_i)+\ln (1-\pi_i)\big)= \sum_{i=1}^n \big (y_i [\ln \pi_i-\ln (1-\pi_i)]+\ln (1-\pi_i)\big)=\sum_{i=1}^n \big (y_i \ln \dfrac{\pi_i}{1-\pi_i}+\ln (1-\pi_i)\big)=\sum_{i=1}^n \big (y_i \ln \dfrac{\pi_i}{1-\pi_i}\big)+\sum_{i=1}^n \ln (1-\pi_i)$

In the 3rd line, I took out $y_i$ first and use the fact that $\ln a- \ln b=\ln (\dfrac{a}{b})$.

$\endgroup$
  • $\begingroup$ Hi Stat, please note the tag help on the self-study tag (which says that policy on such questions is to provide 'helpful hints'). I think the essential insight to do with the manipulation of the log() terms could have been conveyed without effectively doing the whole derivation. $\endgroup$ – Glen_b Mar 24 '13 at 22:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.