33
$\begingroup$

I want to know why logistic regression is called a linear model. It uses a sigmoid function, which is not linear. So why is logistic regression a linear model?

$\endgroup$
2
  • 6
    $\begingroup$ The logit of $\pi$ (the log of the odds) is linear in the parameters, but people don't refer to logistic regression as linear as far as I know. Can you cite who has said this? $\endgroup$ Mar 3 '14 at 18:05
  • 2
    $\begingroup$ @gung-ReinstateMonica For example, in the Deep Learning book at page 169 (deeplearningbook.org/contents/mlp.html). In the book they note "Linear models, such as logistic regression and linear regression, are appealing....." I think they meant Generalized Linear Model for logistic regression. $\endgroup$
    – YOUNG
    Nov 27 '19 at 6:45
41
$\begingroup$

The logistic regression model is of the form $$ \mathrm{logit}(p_i) = \mathrm{ln}\left(\frac{p_i}{1-p_i}\right) = \beta_0 + \beta_1 x_{1,i} + \beta_2 x_{2,i} + \cdots + \beta_p x_{p,i}. $$ It is called a generalized linear model not because the estimated probability of the response event is linear, but because the logit of the estimated probability response is a linear function of the predictors parameters.

More generally, the Generalized Linear Model is of the form $$ \mathrm{g}(\mu_i) = \beta_0 + \beta_1 x_{1,i} + \beta_2 x_{2,i} + \cdots + \beta_p x_{p,i}, $$ where $\mu$ is the expected value of the response given the covariates.

Edit: Thank you whuber for the correction.

$\endgroup$
6
  • 8
    $\begingroup$ If you were to write "generalized linear" instead of "linear" and parameters instead of predictors, this would be correct. (Many logistic regression models are not linear in the predictors. For instance, no logistic regression with an interaction term will be linear in the predictors.) $\endgroup$
    – whuber
    Mar 3 '14 at 18:42
  • $\begingroup$ You are correct, thank you. I've updated my answer to reflect this. $\endgroup$
    – P Schnell
    Mar 3 '14 at 18:55
  • $\begingroup$ what is Pi there? $\endgroup$
    – aerin
    Sep 14 '17 at 6:18
  • $\begingroup$ @whuber Why would an interaction term in a logistic regression wreck the linearity in the parameters when that is not the case for a linear regression? (I assume you mean predicting log-odds. If you mean predicting the probability, then I am confused how logistic regression could be linear in the parameters at all.) $\endgroup$
    – Dave
    Oct 9 '20 at 16:20
  • 2
    $\begingroup$ @Dave In my parenthetical comment I wasn't writing about linearity of the parameters: it was explicitly about the predictors. Nowhere in the comment did I claim linearity of logistic regression models in either parameters or predictors. $\endgroup$
    – whuber
    Oct 9 '20 at 16:24
9
$\begingroup$

Logistic regression uses the general linear equation $Y=b_0+∑(b_i X_i)+\epsilon$. In linear regression $Y$ is a continuous dependent variable, but in logistic regression it is regressing for the probability of a categorical outcome (for example 0 and 1).

The probability of $Y=1$ is: $$ P(Y=1) = {1 \over 1+e^{-(b_0+\sum{(b_iX_i)})}} $$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.