0
$\begingroup$

I'm trying to understand the importance of the log of odds in logistic regression, typically I understand the formulation but unable to get the intuition behind it. Linear regressor gives a continuous value for prediction while Logistic regression is used to classify multi-label classes.

According to the formula:

y = Xβ+u

wherein the case of logistic regression y is the probability measure

given:

log(p / 1−p)= Xβ ---(1)

And eq (1) is called logits, why is this so, why can't it be just raw values and further passed to softmax will convert it into probability measure anyway? In the case of multi-label classification we just want to know the probability values corresponding to the respective classes which tell us what's the most probable mapping from given feature vectors to class labels.

I'm just not able to process the existence of logit. Why is that helpful/required, what's the usefulness of taking log of odds, how LHS equals RHS, what does it signifies or why does that exist in this case? I'm not from a statistical background and beginner in ml, so wanted a deep insight. If I'm missing something, please clarify!

$\endgroup$
1
$\begingroup$

The logit is used here as the link function. The RHS of (1) can range over the real numbers, and this is the reason for the logarithm, to ensure that the LHS also ranges over the reals—without the logarithm it would only range over the non-negative real numbers. Further, this link function is the canonical link function. Other link functions include the probit and complementary log-log functions. See the generalized linear model Wikipedia page.

Although the logit has a name, the interpretation of the coefficients are easier in a sense when you take the exponential of each side of (1). Then the LHS is the odds, which, if the regressor variable is continuous can be used to give an odds ratio interpretation of the regressor. There is a section about this on the logistic regression Wikipedia article.

Equation (1) can also be rearranged to give a formula for the probability. So the logit on the LHS is used as the link function but once the logistic regression is fitted it is often rearranged to either get the odds or probabilities.

$\endgroup$
4
  • $\begingroup$ the logit on the LHS is used as the link function but once the logistic regression is fitted it is often rearranged to either get the odds or probabilities what is this supposed to mean? $\endgroup$ Oct 22 '20 at 14:37
  • $\begingroup$ So according to the first part logs were introduced to hold the equality, to link probability to real-valued function, good point! so that's why we reverse the function as softmax at the end to get the probabilities back? Did I get it right? $\endgroup$ Oct 22 '20 at 14:57
  • $\begingroup$ @offset-null1 the logit, the LHS of (1), is often rearranged to either get the odds (be exponentiating) or rearranging further to get a formula for probability. In terms of the softmax question, yes its used to get the probability of each class, there is more on this in the multinomial logistic regression Wikipedia article. $\endgroup$ Oct 22 '20 at 16:06
  • $\begingroup$ Thanks for answering! $\endgroup$ Oct 22 '20 at 23:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.