Least squares logistic regression I have seen it claimed in Hosmer & Lemeshow (and elsewhere) that least squares parameter estimation in logistic regression is suboptimal (does not lead to a minimum variance unbiased estimator).  Does anyone know other books/articles that show/discuss this explicitly?  Google and my personal library have not helped me here...
 A: It is a well known fact that if the model is parametric (that is, specified completely up to a finite number of unknown parameters), and certain regularity conditions hold, then Maximum Likelihood estimation is asymptotically optimal (in the class of regular estimators). I have doubts about the UMVUE concept, since MLE rarely gives unbiased estimators.
The question of why is MLE optimal is rather tough, you can check for example Van der Vaart's "Asymptotic Statistics", chapter 8.
Now it is known that least squares coincides with MLE if and only if the distribution of error terms in the regression is normal (you can check the OLS article on Wikipedia). Since in logistic regression the distribution is not normal, LS will be less efficient than MLE.
A: In ordinary linear regression maximizing the likelihood is
equivalent to minimizing the sum of squared errors across
the board (and consequently the estimated variance of
errors)
I In logistic regression, the errors are not expected to have
the same variance: we should have high variance for p
near .5, lower variance towards the extremes
I Leads to (iteratively) (re)weighted least squares (IRWLS)
method, where errors are penalized more where we expect
less variance around p
