I've been revising the concepts of logistic regression and suddenly realized that the probability function of the logistic model in the book ISL looks absolutely different from other sources. For ISL logistic function looks this way:

$P(x) = \frac{e^{(\beta_0 + \beta_1x)}}{1 + e^{(\beta_0 + \beta_1x)}}$

(sorry for my lack of knowledge of acceptable formatting). For other sources (especially, courses) logistic function looks another way:

$P(x) = \frac{1}{1 + e^{(\beta_0 + \beta_1x)}}$

(so-called sigmoid function). Moreover, 2 different approaches are used to estimate the coefficients $\beta_0$ and $\beta_1$: for the first equation we're maximizing the likelihood function, while for another we're minimizing the cost function for this model.

Can you please explain me the difference (in simple words) and tell what approach should be used for what case?

  • 2
    $\begingroup$ You're missing a minus sign. In that casw if you divide by the exponentials on both sides of the division you get the same thing. Look at $\endgroup$ – while Jul 7 '16 at 22:59
  • $\begingroup$ Oops, i was going to say Wikipedia. en.m.wikipedia.org/wiki/Sigmoid_function $\endgroup$ – while Jul 7 '16 at 22:59
  • $\begingroup$ Oh, thanks a lot, I've really missed that small -1 in the exponent. $\endgroup$ – olejnik_ Jul 7 '16 at 23:13
  • $\begingroup$ You should really edit that correction into your post! $\endgroup$ – kjetil b halvorsen Sep 6 '17 at 13:19

(Summarizing answer in comments) They are really the same, after you put in that minus sign you lost. Just multiply in numerator and denominator of first form with $\exp(-\beta_0-\beta_1 x)$.

Logistic regression is usually estimated by maximizing the likelihood function, but the negative of the loglikelihood function is often (especially in machine learning circles) called the log-loss. It is really the same thing, only wrapped differently.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.