# Least squares logistic regression [duplicate]

I have seen it claimed in Hosmer & Lemeshow (and elsewhere) that least squares parameter estimation in logistic regression is suboptimal (does not lead to a minimum variance unbiased estimator). Does anyone know other books/articles that show/discuss this explicitly? Google and my personal library have not helped me here...

• Are you asking for a list of such books? It is common knowledge that OLS regression is inappropriate with a binary response variable, if you need a citation, H&L should be good enough. What is the point of this question? – gung - Reinstate Monica Mar 19 '12 at 21:59
• Gung, As the OP says, H&L merely state that when LS is used, "the estimators no longer have [a number of desirable properties]" [2nd Ed. pp 5-6]. This statement is vague, it is offered without further explanation, and no citation is provided. This question does not solicit a list, but rather seeks credible support for an otherwise unfounded assertion. I think it's a fair question and gladly vote it up. – whuber Mar 19 '12 at 22:57
• Gung: whuber stated most of what I would have stated. I also want to make it clear that I am not talking about fitting a regression model with normal error (to data with binary response). I am talking about using the usual logistic regression model, except estimating the parameters via least squares instead of via maximum likelihood methods. – user9968 Mar 20 '12 at 1:10
• For independent binary outcomes, if the true mean response $\mathbb{E}[Y|X]$ follows a logistic-linear model, then optimal tests & estimates are indeed given by logistic regression. If the truth isn't logistic-linear, you don't the same uniform optimality. You do, however, get semi-parametric efficiency from logistic regression, among the class of linear estimation equations, following from Godambe and Heyde's theory of optimal EEs, exploiting the known mean-variance relationship for binary outcomes. LS estimation is worse in both cases - which doesn't make it universally worse. – guest Mar 20 '12 at 6:56
• One reason why least squares is sub-optimal, is that for any binary random variable, $X$, we have $E[X^k]=E[X]=Pr(X=1)$ for all $k$. This means that the variance of $X$ is equal to $E[X](1-E[X])$. So you would at least need to do iterative weighted least squares, as the variance is not independent of the mean (no homoscedasticity). This is basically what logistic regression does, with the added feature that it forces all predictions to lie in $[0,1]$. – probabilityislogic Apr 4 '12 at 0:43

• Your last paragraph is not quite right. If you are talking about regression, it means that the goal of analysis is $E[y|x]$. Logistic regression, instead, is after the conditional probability ${\rm Pr}[y=1|x]$. It is, of course, a matter of coincidence that $E[y|x]={\rm Pr}[y=1|x]\times 1+ {\rm Pr}[y=0|x]\times0={\rm Pr}[y=1|x]$, so you can just as well run a nonlinear regression with $\Lambda(x'\beta)$ as the conditional mean function, and that's what H&L probably had in mind. – StasK Apr 26 '12 at 2:42