I have read about logistic regression on Quora and also from different online source and they said that,

$$logit = b_0+b_1x$$


$logit = \log(\frac{p}{1-p}) = \log (\frac{probability-of-event-happening)}{probability-of-event-not-happening)} = \log (Odds)$

$or,\log(\frac{p}{1-p}) = b_0+b_1x $

My question is we know output, $y = b_0+b_1x $ So how $y$ become $\log(\frac{p}{1-p})$ or why $\log(\frac{p}{1-p})$ is equal to $b_0+b_1x $

Does $\log(\frac{p}{1-p})$ produced the same output as $y$

Edit: Finally I have found my answer which is given in @Peter_Flom's comment. But another question came into my mind-

What is the proof that $p = \frac{exp(b_0+b_1x)}{exp(b_0+b_1x) + 1}$is the probability in Logistic regression?

  • $\begingroup$ It could be easier if the explanation is given using an example that you know, do you have a case where you would like to apply logistic regression ? $\endgroup$ – user83346 Sep 25 '17 at 15:57
  • $\begingroup$ @Glen_b I meant something different and made myslen not clear, sorry. Deleted the comment. $\endgroup$ – Tim Sep 26 '17 at 8:21
  • $\begingroup$ The answer to your edit is straightforward algebra: solve $\log(p/(1-p))=b_0+b_1 x$ for $p$. If indeed Peter Flom's answer solves your problem then (a) upvote it; (b) mark it as accepted; and (c) do not modify the original question! $\endgroup$ – whuber Sep 26 '17 at 17:15
  • $\begingroup$ At first I was asking why $\log(\frac{p}{1-p}) == y$ none of them can give the exact answer. But Peter Flom's help me to find some basic about Logistic Regression. I have given the answer up vote. I'll mark the answer accepted in my own way. And did you understand my edited question really. How a straightforward algebra proves that $p$ is the probability. @whuber $\endgroup$ – Jobayer sheikh Sep 26 '17 at 17:35
  • $\begingroup$ For your edit: In my opinion there is no such proof, it is an assumption underlying the logistic regression model, you assume that , for an element of your population, the probability of success depends on some variables (your $x$) and that the dependence has the functional form $p=\frac{1}{1+e^{-(b_0+b_1x)}}$. You assume that this is the functional relationship between the probability of success and the $x$. This could be the case because whatever $x$, this $p$ will be between 0 and 1. You might assume other functional relationships, if the result is something between 0 and 1. (cont) $\endgroup$ – user83346 Sep 27 '17 at 6:49

In binomial logistic regression, the dependent variable takes only two values, which can be coded 0 and 1. So, it can't be that $Y = b_0 + b_1X$ because that could result in any value. The logit transformation solves this problem.

We take the odds to make the value continuous. We take the odds ratio to get a parameter estimate and we take the log of that ratio to make the variable range from negative infinity to infinity and be symmetric around 0 instead of 1.

  • $\begingroup$ Does Log (p/1-p) produced the same output as y or may be I missed some basic. $\endgroup$ – Jobayer sheikh Sep 25 '17 at 12:04
  • $\begingroup$ Doing regression on Y when Y is dichotomous doesn't work right. $\endgroup$ – Peter Flom Sep 25 '17 at 12:38
  • $\begingroup$ The odds are no more or less continuous than the $p$'s. Did you mean something other than "continuous" there? $\endgroup$ – Glen_b Sep 26 '17 at 1:34
  • 1
    $\begingroup$ $\log(\frac{p}{1-p})$ can also produce positive infinity or negative infinity as $y = b_0+b_1x$. If $p = 1$ then $\log(\frac{p}{1-p}) = +infinity$ or If $p = 0$ then $\log(\frac{p}{1-p}) = -infinity$ where $p = \frac{exp(b_0+b_1x)}{exp(b_0+b_1x) + 1}$. If we solve this equation we get, $\log(\frac{p}{1-p}) = \log(\frac{\frac{exp(b_0+b_1x)}{exp(b_0+b_1x) + 1}}{1-\frac{exp(b_0+b_1x)}{exp(b_0+b_1x) + 1}})=\log(exp(b_0+b_1x)) = b_0+b_1x$ what's why we can write $\log(\frac{p}{1-p}) = b_0+b_1x$. Now another question arise in my mind how $p = \frac{exp(b_0+b_1x)}{exp(b_0+b_1x) + 1}$ is probability. $\endgroup$ – Jobayer sheikh Sep 26 '17 at 4:58

If you just modeled $Y$ directly that is called a linear probability model--if you believe that covariates affect the probability in a linear way, that wouldn't be a ridiculous thing to do (despite Peter Flom's blanket statement that "regression on Y when Y is dichotomous doesn't work right"; also, just so you don't misunderstand---the model $\beta_0 + \beta_1 X$ is for the expected value of Y, which is a probability, so prior comments about $Y$ not being allowed to take on values other than 0/1 are irrelevant). You could fit a linear probability model using least squares (although be careful with the p-values) or using the binomial likelihood directly, although numerical problems can arise from non-finite likelihoods, which you can usually handle with good starting values (and is less of a problem when you have a good sample size).

People usually use logistic regression for this situation, modeling the probability on the log odds scale, because of the numerical stability and the convenient interpretation of the coefficients as log odds ratios.

  • $\begingroup$ It's not irrelevant, I suggest, that in principle the predictions from the linear probability model could lie outside [0, 1], although in practice this doesn't seem as common -- within the range of the data -- as some warnings imply. $\endgroup$ – Nick Cox Sep 25 '17 at 14:34
  • 1
    $\begingroup$ If you're also stats.stackexchange.com/users/109178/gammer please merge your accounts. $\endgroup$ – Nick Cox Sep 25 '17 at 14:35
  • 1
    $\begingroup$ @NickCox, I agree that the bounding between 0 and 1 is likely not a big practical problem as long as you're not extrapolating, especially if the sample size is reasonable. Also, yes that account is mine--just got logged out (new computer) and forgot the email/password I signed up with. Not sure if I can merge accounts. I'm ok using this one. $\endgroup$ – gammer Sep 25 '17 at 17:52
  • 2
    $\begingroup$ stats.stackexchange.com/help/merging-accounts $\endgroup$ – Nick Cox Sep 25 '17 at 18:13

In general, regression on $\log(\frac{p}{1-p})$ gives a different answer than regression on Y. So, no, the results you will receive will be different.

As the former answer already states, if your $Y$ takes only two values you usually do not want to model $Y$ directly - that is just because it is way easier to analyze continuous functions. So what you do is you assume $Y$ is Bernoulli (well it is, it only takes two values) and model $\frac{p}{1-p}$, where $p$ is $\mathbb{P}(Y=1)$. This way you can model your $Y$.

But, to reiterate: Regression on $\frac{p}{1-p}$ produces a different output than regression on $Y$.

  • 1
    $\begingroup$ With several answers at about the same time, people don't want to have to decode which "the former answer" is. Best to cross-reference by citing the author. $\endgroup$ – Nick Cox Sep 25 '17 at 14:30
  • $\begingroup$ Unfortunately $\log(\frac{p}{1-p})$ produces the same result as $y$ but can you explain why $p = \frac{exp(b_0+b_1x)}{exp(b_0+b_1x) + 1}$ is probability.. $\endgroup$ – Jobayer sheikh Sep 26 '17 at 11:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.