Have I understood logictic regression output correctly? Logit, odds ratio and probability ratio

Question

Here is an example model using diamonds dataset:

library(tidyverse)
library(caret)
library(broom)

# data
my_diamonds <- diamonds
my_diamonds <- my_diamonds %>% 
  mutate(target = ifelse(cut == "Fair" | cut == "Good", 0, 1))


# statistical modeling with logistic regression via caret
## tuning & parameters
set.seed(123)
train_control <- trainControl(
  method = "cv",
  number = 5,
  savePredictions = TRUE,
  verboseIter = TRUE,
  classProbs = TRUE,
  summaryFunction = prSummary
)

# fit model
linear_model = train(
  x = select(my_diamonds, c(price, x, y, z)), 
  y = my_diamonds$target %>% make.names() %>% factor(levels = c("X1", "X0")),
  trControl = train_control,
  method = "glm", # logistic regression
  family = "binomial",
  metric = "AUC" # actually prAUC
)

# get coefficients and various ratios
model_df <- broom::tidy(linear_model$finalModel) %>% 
  select(term, estimate) %>% 
  mutate(odds_ratio = exp(estimate))

Here is the output of model_df: model_df

# A tibble: 5 x 3
  term         estimate odds_ratio
  <chr>           <dbl>      <dbl>
1 (Intercept) -5.81        0.00299
2 price       -0.000201    1.000  
3 x           -1.72        0.179  
4 y           -2.45        0.0867 
5 z            7.99     2959.     

Within the context of inference and understanding, can I infer the following:

1. For inference, the estimate is not so interesting, it's the logit which is scaled log odds between 0 and 1. Typically, of more interest is the log odds... True or False?

2. The log odds, calculated above with exp(estimate) are the change in odds relative to the intercept of each coefficient. Example, in this model price has a log odds ratio of exactly 1, so the change in odds per one unit change in price is unchanged, since it changes by a factor of 1 only. i.e. if price is 10 then the odds of target being 1 are still just 0.00299 all else being equal. If the log odds for price were 2 and not 1, then the odds of target being 1 when price is 10 would be (0.00299*2)^10. Is this correct?

3. Each time input variable x increases by one, the log odds of target being 1 increases by 0.179 * -1.72 = −0.30788.

4. For understanding actual probabilities, I was initially tempted to do this:

# don't do this because the probability ratio is not linear, must be applied to a given observation not the coefficients model_df_dont_do_this <- model_df %>% mutate(probs_ratio = exp(estimate)/(1 + exp(estimate)))

Looks like this:

model_df_dont_do_this
# A tibble: 5 x 4
  term         estimate odds_ratio probs_ratio
  <chr>           <dbl>      <dbl>       <dbl>
1 (Intercept) -5.81        0.00299     0.00298
2 price       -0.000201    1.000       0.500  
3 x           -1.72        0.179       0.152  
4 y           -2.45        0.0867      0.0798 
5 z            7.99     2959.          1.000  

However, from researching on this site and elsewhere, this is nonsensical since the probability ratio is not linear and can only be applied to a given observation, not to the coefficients. Is that correct?

I'm seeking validation on my understanding. Have I understood logistic regression correctly? If not, where have I misunderstood?

Answer 1

You are correct in your statement "this is nonsensical since the probability ratio is not linear and can only be applied to a given observation, not to the coefficients". Even what you call "odds ratio" are not quite coefficients, though they are useful

I cannot actually get your linear_model to work, though

fit1 <- glm(target ~ price + x + y + z, family=binomial, data=my_diamonds)
fit1$coefficients
exp(fit1$coefficients)
exp(fit1$coefficients) / (1 + exp(fit1$coefficients))

produces the same coefficients as you have up to rounding, though with the "log-odds coefficients" signs reversed, the "odds coefficients" as the reciprocals of yours, and the "probability coefficients" $1$ minus your values (probably something to do with the translation to factors and their use, so your log-odds, odds and probabilities may be that the diamonds are Fair or Good). That point does not affect your actual question

For the first diamond, with a price of $326$, $x=3.95$, $y=3.98$, $z=2.43$, you can say that the model's log-odds are about $$-5.81 -0.000201 \times 326 -1.72 \times 3.95 -2.45 \times 3.98 + 7.99 \times 2.43 \approx -3$$ and the model's odds for this diamond are about $$0.00299 \times 1.000^{326} \times 0.179^{3.95} \times 0.0867^{3.98} \times 2959^{2.43} \approx 0.05$$ which is not much of a surprise as $e^{-3}\approx 0.05$

But there is no equivalent simple interpretation on how to use your column labelled probs_ratio to combine with the known values for this diamond to give a model probability also about $0.05$