Interpreting Probability of Logistic Regression

Question

The accepted answer at this post is consistent with my understanding:

Intepreting 'one-unit' Change in Logistic Regression

Especially where it says: To find the change in terms of the proportions that are modelled you need to:

Get the log(odds) estimate. Exponentiate it to get the odds. Get the new proportions as: Prnew=odds1+odds. (This follows from the equality shown above.)

My questions is whether you can make a statement about what happens to probability based on a one unit increase of the coeficient.

Since prob = e^(bo + b1x1 + b2x2) / ( 1 + e^(bo + b1x1 + b2x2)) it seems that it is not linear, so that you couldn't.

In addition, there is this comment on the same post: "Notice that the change to the predicted proportions is not quantized in the same way as the log(odd) transform is not linear."

A friend in an analytics course, though, is telling me that perhaps we could, get the change in odds by taking the log, then get the change in probability by evaluating the odds and using odds / (1 + odds) for probability.

Is this possible or not?

Answer 1

You can get the change in odds by taking the anti-log of the coefficients, but the effect is not additive but multiplicative.

$ \ln\left(\frac{\mathrm{Pr}}{1-\mathrm{Pr}}\right) = \ln(\mathrm{odds}) = \beta_0 + \beta_1 x_1 + \beta_2 x_2 $

$ \exp(\ln(\mathrm{odds})) = \exp(\beta_0 + \beta_1 x_1 + \beta_2 x_2) $

$ \mathrm{odds} = \exp(\beta_0) \times \exp(\beta_1 x_1) \times \exp(\beta_2 x_2) $

If we now look at what happens when $x_1$ increases by $1$, we get:

$ \mathrm{odds} = \exp(\beta_0) \times \exp(\beta_1 (x_1+1)) \times \exp(\beta_2 x_2) $

$ \mathrm{odds} = \exp(\beta_0) \times \exp(\beta_1 x_1 + \beta_1) \times \exp(\beta_2 x_2) $

$ \mathrm{odds} = \exp(\beta_0) \times \exp(\beta_1 x_1) \times \exp(\beta_1) \times \exp(\beta_2 x_2) $

So if we compute the ratio of the old and the new odds:

$ \frac{\mathrm{odds_{new}}}{\mathrm{odds_{old}}} = \frac{\exp(\beta_0) \times \exp(\beta_1 x_1) \times \exp(\beta_1) \times \exp(\beta_2 x_2)}{\exp(\beta_0) \times \exp(\beta_1 x_1) \times \exp(\beta_2 x_2)} = \exp(\beta_1) $

$\exp(\beta_1)$ is called an odds ratio. Lets say you find an odds ratio of 1.2, then you would interpret that as a unit increase in $x_1$ is associated with an increase in the odds by a factor of 1.2, i.e. the odds needs to be multiplied by 1.2, which corresponds to an increase of 20%. Similarly if we find an odds ratio of 0.8 a unit increase in $x_1$ is associated with a change in the odds by a factor of 0.8, i.e. the new odds is 20% smaller. So no effect is represented by an odds ratio of 1.

If you want an additive effect on probabilities you can look at marginal effects. Personally, I consider those as a two-step approximation of a linear probability model. Why would you do something in two steps if you can do so in one? My order of preference would be to use logistic regression and interpret odds ratios, followed by a linear probability model, and never ever ever ever even think about marginal effects. But this ordering is somewhat controversial...