3
$\begingroup$

On page 137 in ISLR 2nd Edition, there is an example of a logistic regression model using a binary variable (1= yes student, 0= not a student) to predict if this person would be more or less likely to default. At the bottom, it shows when you plug in the coefficients and the respective dummy variable, the student has a 4.31% probability of default and the non student has a 2.92% probability of default.

I have 2 follow up questions:

  1. Is it fair to say that the student has a 1.39% higher probability of default than the non-student? (=4.31-2.92). If no, why not?

  2. If the above is fair, why do we even deal with odds ratios? The table shows that the student coefficient is 0.4049, which means e^0.4049 = 1.49915258, which means "There is a 49.9% increase in odds of a default occurring given that it is a student". Isn't it much easier to just say the probability of a student defaulting is 1.39% higher than a non-student?

enter image description here


Edit:

Loophole discovered. Looks like this doesn't work for continuous variables (example on pg 136), but OK for dummy variables. Looks like odds ratio relationships can be extrapolated for all 1 unit differences in feature X, but probability differences would only work for 2 specific X value differences:

enter image description here

$\endgroup$

1 Answer 1

6
$\begingroup$
  1. It’s fair to say that, but you might say 1.39 percentage points rather than 1.39%. The former makes it absolutely clear that we’re talking about differences in probabilities.

  2. The natural interpretations of the model coefficients are on the log odds scale. The reason they are on the log odds scale is so that the linear predictor is unbounded (unlike probabilities which are on the unit interval). There are some other reasons for example, effects are more likely to be linear on the log odds scale, but in general the use of odds and log odds is for mathematical convenience and not because they are more interpretable.

EDIT: Your "loophole" is expected behaviour. The property you're wanting is called "linearity". Logistic regression is linear on the log odds scale, so log odds will have the property you seek. However, the sigmoid function is not linear, which explains the difference you've found.

$\endgroup$
4
  • $\begingroup$ So this was not what I was expecting, but this is good news! Sounds like I can just use difference in percentage points going forward for interpretability, especially when I am communicating to non technical stakeholders/in the business world? $\endgroup$
    – Katsu
    Jan 5, 2023 at 4:57
  • 4
    $\begingroup$ @Katsu if you never mentioned an odds ratio to a stakeholder, you would be doing very well for you and for them. $\endgroup$ Jan 5, 2023 at 5:11
  • $\begingroup$ OK i think i found a loophole. Lets say feature X was a continuous variable and not a dummy variable. It seems like the odds ratio can be extrapolated to all 1 unit increases in feature X, but the probability increase can only apply to 2 specific values of feature X? $\endgroup$
    – Katsu
    Jan 5, 2023 at 5:21
  • 2
    $\begingroup$ @Katsu You can always convert the predicted log odds to a probability and take differences on that scale. But you're right, the log odds have the benefit of being able to scale linearly so you know that every 1 unit change in the predictor yields a $\beta$ change in the log odds. This does not apply on the probability scale, but as I mentioned you can just convert to probability scales and report differences for desired levels of the covariates there. $\endgroup$ Jan 5, 2023 at 5:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.