# Why logistic regression cannot be solved by OLS

I know I mess up different things. However I want to get better understanding what is motivation behind logistic regression.

$p(y_i=1|x_i)=(1+e^{-x_iw})^{-1}$

and according to OLS

$S(w)=\sum_{i}^{}(y_i-\hat{y_i})^2$,where $y \in \{1,0\}$.

What if $\hat{y_i}=p(y_i=1|x_i)$ and estimate $w$ by finding minimum to $S(w)$.

Why this is incorrect approach?

• If you want to use OLS techniques, a (much) better way to study this relationship is to regress $x_i$ against $y_i$, rather than $y_i$ against $x_i$. As I recall, Hosmer and Lemeshow even recommend that as a reality check and way to get starting coefficients for fitting a logistic regression. – whuber Mar 5 '15 at 19:28

I think this essentially boils down to what cost function you want to minimize in order to estimate your parameter $w$. Typically, the negative log-likelihood is minimized for parameter estimation, what you have suggested looks like minimizing the Brier score. I think they would give very similar estimates for $w$ (edit: see comments).
• Agree up until the last sentence. They can give very different estimates for $w$. – Neil G Mar 5 '15 at 11:41