2
$\begingroup$

I have a question regarding logistic regressions.

Meyers et al. (2013) say that "It is possible to create a linear relationship between the predictors and odds" (p.533). They are referring to the predictors in a logistic regression. However, they also say, after introducing the idea of log odds, that "This transformation 'bends' the data to fit the sinusoidal curve" (p.533).

I'm left a bit unclear whether the relationships between predictors and the outcome variable is modeled linearly or not? I had assumed logistic regression uses a sinusoidal curve; however, the first quotation makes it sound as though it actually models a linear relationship.

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

Logistic regression is a GLM. The linear ("L") part of GLM relates to the fact that the covariates are included within a linear predictor. However, the linear predictor is transformed by a link function, which may be non-linear, to get to the expected response.

In other words, the relationship between the predictors and the expected response is non linear in this case.

Improve this answer
$\endgroup$
0
$\begingroup$

Logistic Regression (while containing a misleading name) is usually a classifier. This means that is is trying to use 'predictors', as you called it, or input features to classify an outcome variable- most commonly a binary outcome variable. The way Logistic Regression does this is by creating a linear boundary function which separates the space of inputs into two. Then it classifies everything on one side as class A and everything on the other as class B.

To answer your question: Logistic regression does not necessarily find a linear correlation, if that is what you mean, or any linear (or sinusoidal) relationship between the inputs and the output. It simply tries to find the "best" linear boundary function which splits the data into class A and class B.

I will try to explain what "best" means. Best is defined as minimizing a cost function which represents the distance between $$1/(1+e^{-w^{T}x})$$, which is a logistic function, and the true value of the corresponding outcome variable of x (Note that w is what logisitc regression is optimizing for and x is the input).

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Depending on the context, logistic regression is not usually a classifier. It can do much more than that -> e.g. output class probabilities, determine the effects of treatments on the log-odds etc. Also, in the sense of regression the cost function is the log-likelihood function of the response distribution. Suggesting to minimise the distance like the squared distance between the true value and the predicted value will lead to suboptimal variances for fitted class probabilities close to zero and one. $\endgroup$
    – Alex
    Commented Mar 1, 2017 at 22:08
  • $\begingroup$ Why logistic regression is not classification: stats.stackexchange.com/questions/127042/… $\endgroup$
    – kjetil b halvorsen ♦
    Commented Jul 11, 2018 at 21:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.