1
$\begingroup$

So I am asked to fit a logit model using the method of least squares in connection to logistic regression. Let $\pi(x)=\mathrm{P}(Y=1|X=x)$ be the probability of success of a binary response variable $Y$ for a given $x$. Then it is established from the differential equation $d(\pi(x))=\beta\,\pi(x)(1-\pi(x))dx$ that the logistic curve is of the form $\pi(x)=\dfrac{e^{\alpha+\beta x}}{1+e^{\alpha+\beta x}}$.

It is also seen while solving the above differential equation that it is the logit $\ln \left(\frac{\pi(x)}{1-\pi(x)}\right)=\alpha+\beta x$ which is actually linear. So could I use the conventional least square method for this function and obtain the least square estimates $\hat{\alpha},\hat{\beta}$ to substitute for $\alpha,\beta$ in the fitted curve $\hat{\pi(x)}$? Is this what I am supposed to do? Or is there a different principle to be followed for fitting the logit model, something like the weighted least squares?

Improve this question
$\endgroup$
5
  • $\begingroup$ note that even after your transformation of the log-odds, the left hand site (=your "dependent variable") depends on the unknown parameters as well and is, per se, unknown. This implies that there is no way that you can just apply a standard OLS to this line. however, while the parameters of a logistic regression model can in fact be fitted with a iteratively (re-)weighted least squares algorithm, an maximum likelihood estimation is the more standard way to go. $\endgroup$
    – EliKa
    Jun 19, 2017 at 11:04
  • 1
    $\begingroup$ A logit model can be expressed as a GLM, and thus may be solved using least squares. However, the standard OLS will not suffice. If you really aim to go down this route, you should have a look at (re)weighted least squares. I'd be remiss, however, if I did not note that MLE is the more conventional approach in this case. $\endgroup$
    – Nelewout
    Jun 19, 2017 at 11:06
  • 1
    $\begingroup$ @N.Wouda Why do you say that OLS will not suffice? $\endgroup$
    – StubbornAtom
    Jun 19, 2017 at 13:52
  • $\begingroup$ (+1) OLS will work to fit the logistic function, but you're right: weighted LS will work better. $\endgroup$
    – whuber ♦
    Jun 19, 2017 at 14:25
  • 1
    $\begingroup$ The variance of a $0/1$ response with probability $p$ is $p(1-p)$: this makes the responses heteroscedastic. If the predicted values of $p$ change much at all, this becomes an important concern--and one cure is to use the reciprocals of the variances as weights in a least squares regression. The tricky part is that you simultaneously have to estimate $p$ and the weights. $\endgroup$
    – whuber ♦
    Jun 19, 2017 at 17:58

0

Reset to default