2
$\begingroup$

I come to a statement that logistic regression is a non linear problem. How can one show this?

Is it possible to treat logistic discrimination in terms of equivalent linear regression problem?

$\endgroup$
  • 3
    $\begingroup$ "Nonlinear problem" and "nonlinear regression" are completely different things! Which of these two questions are you trying to ask? $\endgroup$ – whuber Sep 4 '18 at 17:54
  • $\begingroup$ Question is " show that logistic regression is a non linear regression problem". $\endgroup$ – ironman Sep 4 '18 at 18:22
  • $\begingroup$ Can you write out a logistic regression function, e.g., $p(y=1) = \dots$? $\endgroup$ – jbowman Sep 4 '18 at 19:33
  • 5
    $\begingroup$ Allow me to elaborate. Logistic regression is linear in the general sense I initially describe in an answer at stats.stackexchange.com/questions/148638. Briefly, it is the archetype of a generalized linear model (GLM). However, estimating its parameters is a nonlinear optimization problem. In which sense do you mean "non linear problem"? And what do you mean by "logistic discrimination" and what sense of "equivalent" might you have in mind in your second question? $\endgroup$ – whuber Sep 4 '18 at 20:39
  • $\begingroup$ @whuber generalized linear model, not a general linear model, right? The usual use of the term "general linear model" does not include logistic regression (see here, noting in particular the "not to be confused with" part). Logistic regression is usually categorized as non-linear regression due to the non-linearity introduced by the link function. $\endgroup$ – guy Sep 4 '18 at 22:52
8
$\begingroup$

Recall that the Logistic regression model is a non linear transformation of $\beta^Tx$

  • Probability of $(Y = 1)$: $p = \frac{e^{\alpha + \beta_1x_1 + \beta_2 x_2}}{1 + e^{ \alpha + \beta_1x_1 + \beta_2 x_2}}$
  • Odds of $(Y = 1)$: $ \left( \frac{p}{1-p}\right) = e^{\alpha + \beta_1x_1 + \beta_2 x_2}$
  • Log Odds of $(Y = 1)$: $ \log \left( \frac{p}{1-p}\right) = \alpha + \beta_1x_1 + \beta_2 x_2$

So to answer your question, Logistic regression is indeed non linear in terms of Odds and Probability, however it is linear in terms of Log Odds.


A simple example

Fitting a logistic regression model on the following toy example gives the coefficients $\alpha = -5.05$ and $\beta = 1.3$

Plotting the probability $P(Y=1)$ as a function of $X$ clearly shows the non linear relationship

enter image description here

The Odds of $Y$ being 1 given $X$ is also non linear

enter image description here

Finally the log odds of $Y$ being 1 is a linear relationship

enter image description here

See here for some more details: Calculating confidence intervals for a logistic regression

$\endgroup$
0
$\begingroup$

For the first statement: logistic regression is used when a variable is dichotomous. Since the variable can assume only value 1 or 0, fitting a line assumes a linear relationship which cannot hold for dichotomous outcomes. It can be proved that the linear probability model will not be efficient and, furthermore, nothing ensures that the estimated dependent variable will be bounded between 0 and 1. The logit can solve these problem.

Please clarify your second statement.

$\endgroup$
  • 1
    $\begingroup$ Suggest editing "fitting a line is counter-productive" to something like "fitting a line assumes a linear relationship which cannot hold for dichotomous outcomes." (To speak more directly to the OP's question. :). And welcome to CV! $\endgroup$ – Alexis Sep 4 '18 at 20:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.