# Logistic Regression is a nonlinear regression problem?

I come to a statement that logistic regression is a non linear problem. How can one show this?

Is it possible to treat logistic discrimination in terms of equivalent linear regression problem?

• "Nonlinear problem" and "nonlinear regression" are completely different things! Which of these two questions are you trying to ask? – whuber Sep 4 '18 at 17:54
• Question is " show that logistic regression is a non linear regression problem". – ironman Sep 4 '18 at 18:22
• Can you write out a logistic regression function, e.g., $p(y=1) = \dots$? – jbowman Sep 4 '18 at 19:33
• Allow me to elaborate. Logistic regression is linear in the general sense I initially describe in an answer at stats.stackexchange.com/questions/148638. Briefly, it is the archetype of a generalized linear model (GLM). However, estimating its parameters is a nonlinear optimization problem. In which sense do you mean "non linear problem"? And what do you mean by "logistic discrimination" and what sense of "equivalent" might you have in mind in your second question? – whuber Sep 4 '18 at 20:39
• @Guy That's fine--but neither I nor the OP ever used "linear model" in that sense; despite the typographical error in my comment, which you kindly pointed out and I corrected, I have been discussing GLMs all along and provided a link to clarify what that was intended to mean. – whuber Sep 5 '18 at 12:47

## 2 Answers

Recall that the Logistic regression model is a non linear transformation of $\beta^Tx$

• Probability of $(Y = 1)$: $p = \frac{e^{\alpha + \beta_1x_1 + \beta_2 x_2}}{1 + e^{ \alpha + \beta_1x_1 + \beta_2 x_2}}$
• Odds of $(Y = 1)$: $\left( \frac{p}{1-p}\right) = e^{\alpha + \beta_1x_1 + \beta_2 x_2}$
• Log Odds of $(Y = 1)$: $\log \left( \frac{p}{1-p}\right) = \alpha + \beta_1x_1 + \beta_2 x_2$

So to answer your question, Logistic regression is indeed non linear in terms of Odds and Probability, however it is linear in terms of Log Odds.

## A simple example

Fitting a logistic regression model on the following toy example gives the coefficients $\alpha = -5.05$ and $\beta = 1.3$

Plotting the probability $P(Y=1)$ as a function of $X$ clearly shows the non linear relationship

The Odds of $Y$ being 1 given $X$ is also non linear

Finally the log odds of $Y$ being 1 is a linear relationship

See here for some more details: Calculating confidence intervals for a logistic regression

For the first statement: logistic regression is used when a variable is dichotomous. Since the variable can assume only value 1 or 0, fitting a line assumes a linear relationship which cannot hold for dichotomous outcomes. It can be proved that the linear probability model will not be efficient and, furthermore, nothing ensures that the estimated dependent variable will be bounded between 0 and 1. The logit can solve these problem.

Please clarify your second statement.