# Logistic regression vs. estimating $\beta$ using linear regression and applying the inverse-logit function

If I want to perform a logistic regression analysis, can I use the linear regression model to compute the values of the parameters and then apply the formula:

$$\frac{1} {1 + \exp-(\beta_0+x_1\beta_1+\ldots +x_i\beta_i)}?$$

• That would be a bad fit. Why would you ever do that? – Memming Sep 27 '13 at 15:12
• why a bad fit ? – T-student Sep 27 '13 at 15:16
• I think what @Memming means to say is that linear regression already supplies some kind of fit in the form of $\beta_0 + x_1\beta_1 + \cdots + x_i\beta_i.$ Applying the logistic function to these values is really going to screw up that fit, likely (but not invariably) making it worse. You can find out more by searching our site for (high-voted) answers about logistic regression and its interpretation: there are lots of them that should give you the insight you seek. – whuber Sep 27 '13 at 15:16
• can you give me a link? please – T-student Sep 27 '13 at 15:18
• I think there's a minus sign missing in the exponent, because the inverse logit function (a.k.a. the logistic function) is $1/(1+e^{-x})$. – Macro Sep 27 '13 at 15:45

This idea is a bit like Platt Scaling (see this previous question and the answers given), which is used to get probabilities from support vector machines and other machine learning methods. You could use this method (although you would need to add a scaling and an offset to the output of the linear regression to make it work as well as possible), however for a linear model, there isn't really much benefit in doing so as it is straight-forward to construct a logistic regression model with similar computational expense, and that will probably give better estimates of probability.

Essentially for most problems it is best to use the statistical method that gives the most direct answer to the question posed, rather than use a simpler (or more complicated) method and post-process the result.

See also Appendix D.2 of Tipping's paper on the Relevance Vector Machine, which explains why Platt scaling may not give as good a result as building a model that estimates the probability of class membership more directly.

A linear regression relates proportions $p$ linearly to predictors $x_1, x_2, \dots$: $$p = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + \dots$$ while a logistic regression relates log-odds linearly to $x_1, x_2, \dots$: $$\log (p/(1-p)) = \alpha_0 + \alpha_1 x_1 + \alpha_2 x_2 + \dots$$ Now, if you want to predict log-odds using linear regression (which is quite unnatural), you would clearly need to apply the logit function on predictions $\hat p$ from the linear regression, not the inverse logit as you propose. So my clear answer is "no, you should not use your formula".

You need to estimate a logistic regression by maximum likelihood. The following link shows the likelihood function along with empirical maximization using Newton's method.