Here is an example model using diamonds dataset:
library(tidyverse)
library(caret)
library(broom)
# data
my_diamonds <- diamonds
my_diamonds <- my_diamonds %>%
mutate(target = ifelse(cut == "Fair" | cut == "Good", 0, 1))
# statistical modeling with logistic regression via caret
## tuning & parameters
set.seed(123)
train_control <- trainControl(
method = "cv",
number = 5,
savePredictions = TRUE,
verboseIter = TRUE,
classProbs = TRUE,
summaryFunction = prSummary
)
# fit model
linear_model = train(
x = select(my_diamonds, c(price, x, y, z)),
y = my_diamonds$target %>% make.names() %>% factor(levels = c("X1", "X0")),
trControl = train_control,
method = "glm", # logistic regression
family = "binomial",
metric = "AUC" # actually prAUC
)
# get coefficients and various ratios
model_df <- broom::tidy(linear_model$finalModel) %>%
select(term, estimate) %>%
mutate(odds_ratio = exp(estimate))
Here is the output of model_df: model_df
# A tibble: 5 x 3
term estimate odds_ratio
<chr> <dbl> <dbl>
1 (Intercept) -5.81 0.00299
2 price -0.000201 1.000
3 x -1.72 0.179
4 y -2.45 0.0867
5 z 7.99 2959.
Within the context of inference and understanding, can I infer the following:
For inference, the estimate is not so interesting, it's the logit which is scaled log odds between 0 and 1. Typically, of more interest is the log odds... True or False?
The log odds, calculated above with
exp(estimate)
are the change in odds relative to the intercept of each coefficient. Example, in this model price has a log odds ratio of exactly 1, so the change in odds per one unit change in price is unchanged, since it changes by a factor of 1 only. i.e. if price is 10 then the odds of target being 1 are still just 0.00299 all else being equal. If the log odds for price were 2 and not 1, then the odds of target being 1 when price is 10 would be (0.00299*2)^10. Is this correct?Each time input variable x increases by one, the log odds of target being 1 increases by 0.179 * -1.72 = −0.30788.
For understanding actual probabilities, I was initially tempted to do this:
# don't do this because the probability ratio is not linear, must be applied to a given observation not the coefficients
model_df_dont_do_this <- model_df %>% mutate(probs_ratio = exp(estimate)/(1 + exp(estimate)))
Looks like this:
model_df_dont_do_this
# A tibble: 5 x 4
term estimate odds_ratio probs_ratio
<chr> <dbl> <dbl> <dbl>
1 (Intercept) -5.81 0.00299 0.00298
2 price -0.000201 1.000 0.500
3 x -1.72 0.179 0.152
4 y -2.45 0.0867 0.0798
5 z 7.99 2959. 1.000
However, from researching on this site and elsewhere, this is nonsensical since the probability ratio is not linear and can only be applied to a given observation, not to the coefficients. Is that correct?
I'm seeking validation on my understanding. Have I understood logistic regression correctly? If not, where have I misunderstood?