# Logistic Regression is a nonlinear regression problem?

I come to a statement that logistic regression is a non linear problem. How can one show this?

Is it possible to treat logistic discrimination in terms of equivalent linear regression problem?

• "Nonlinear problem" and "nonlinear regression" are completely different things! Which of these two questions are you trying to ask? – whuber Sep 4 '18 at 17:54
• Question is " show that logistic regression is a non linear regression problem". – ironman Sep 4 '18 at 18:22
• Can you write out a logistic regression function, e.g., $p(y=1) = \dots$? – jbowman Sep 4 '18 at 19:33
• Allow me to elaborate. Logistic regression is linear in the general sense I initially describe in an answer at stats.stackexchange.com/questions/148638. Briefly, it is the archetype of a generalized linear model (GLM). However, estimating its parameters is a nonlinear optimization problem. In which sense do you mean "non linear problem"? And what do you mean by "logistic discrimination" and what sense of "equivalent" might you have in mind in your second question? – whuber Sep 4 '18 at 20:39
• @whuber generalized linear model, not a general linear model, right? The usual use of the term "general linear model" does not include logistic regression (see here, noting in particular the "not to be confused with" part). Logistic regression is usually categorized as non-linear regression due to the non-linearity introduced by the link function. – guy Sep 4 '18 at 22:52

Recall that the Logistic regression model is a non linear transformation of $\beta^Tx$

• Probability of $(Y = 1)$: $p = \frac{e^{\alpha + \beta_1x_1 + \beta_2 x_2}}{1 + e^{ \alpha + \beta_1x_1 + \beta_2 x_2}}$
• Odds of $(Y = 1)$: $\left( \frac{p}{1-p}\right) = e^{\alpha + \beta_1x_1 + \beta_2 x_2}$
• Log Odds of $(Y = 1)$: $\log \left( \frac{p}{1-p}\right) = \alpha + \beta_1x_1 + \beta_2 x_2$

So to answer your question, Logistic regression is indeed non linear in terms of Odds and Probability, however it is linear in terms of Log Odds.

## A simple example

Fitting a logistic regression model on the following toy example gives the coefficients $\alpha = -5.05$ and $\beta = 1.3$

Plotting the probability $P(Y=1)$ as a function of $X$ clearly shows the non linear relationship

The Odds of $Y$ being 1 given $X$ is also non linear

Finally the log odds of $Y$ being 1 is a linear relationship

See here for some more details: Calculating confidence intervals for a logistic regression

For the first statement: logistic regression is used when a variable is dichotomous. Since the variable can assume only value 1 or 0, fitting a line assumes a linear relationship which cannot hold for dichotomous outcomes. It can be proved that the linear probability model will not be efficient and, furthermore, nothing ensures that the estimated dependent variable will be bounded between 0 and 1. The logit can solve these problem.