2
$\begingroup$

I don't entirely understand why logistic regression is based on an exponential function.

The sigmoid function seems to assume that as the dependent variables increase, the independent variable starts to increase exponentially. How do we know this is true? How do we know that this isn't a linear relationship?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ stats.stackexchange.com/questions/88603/… $\endgroup$
    – Sycorax ♦
    Commented Aug 3, 2017 at 1:37
  • 7
    $\begingroup$ Note that the model is for a probability $P(Y=1|\mathbf{X}=\mathbf{x})$ which must lie between 0 and 1. A linear function must eventually pass outside those limits, giving impossible probabilities. The logit is an example of a link function - by far the most popular - that stays within those limits. There are a variety of other functions that can be used (and indeed, both probit and complementary-log-log links are sometimes used instead of the logit link). $\endgroup$
    – Glen_b
    Commented Aug 3, 2017 at 2:06
  • $\begingroup$ @Glen_b nice concise explanation. I have to disagree with the logit being by far the most popular claim, though. In econometrics we usually resort to Probit link function. I think, that's because of the easier interpretability of the regression coefficients $\endgroup$
    – KenHBS
    Commented Aug 3, 2017 at 9:47
  • 2
    $\begingroup$ @Ken econometricians do use the probit a lot (and if I was talking only about the work done in econometrics you'd have a point), but econometricians are a small fraction of the people doing this kind of modelling. In applications across all areas I'd have guessed it would be over 90% logistic and less than 10% everything else put together (which would be almost all probit). [For questions here it's about 80/20 (logistic + logit to probit) but a somewhat higher proportion of new logistic questions would be closed as duplicates than would be the case for probit questions.] $\endgroup$
    – Glen_b
    Commented Aug 3, 2017 at 9:49
  • $\begingroup$ @Glen_b I understand that logit allows the function to stay within the 0-1 limits. However, that still doesn't answer my original question. How do we know that the independent variable increases exponentially, and why is it safe to assume that? $\endgroup$
    – bugsyb
    Commented Aug 3, 2017 at 23:26

1 Answer 1

Reset to default
6
$\begingroup$

[I didn't answer this before because I thought it would be a duplicate but I didn't locate a suitable one; I'll base a brief answer off my comments, at least until such a thread is located.]

Note that the model is for a probability $P(Y=1|\mathbf{X}=\mathbf{x})$ which must lie between 0 and 1.

A linear function must eventually pass outside those limits, giving impossible probabilities. That's usually undesirable.

The logit is an example of a link function - the most popular - that stays within those limits.

There are a variety of other functions that can be used for example, both probit and complementary-log-log links are sometimes used instead of the logit link for conditionally binomial models.

While convenient from several viewpoints, it's no more safe to assume it's specifically logistic than to assume it's anything else that satisfies the same restrictions, outside of the fact that it's often a reasonable approximation$-$as the famous saying goes, all models are wrong, but some are useful.

It makes about as much sense as to assume a relationship is linear in ordinary regression. It's convenient and often a good approximation, but typically it would be unwise to think the relationships are actually exactly linear.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.