3
$\begingroup$

Take the following logistic regression: $$\log(o) = \beta_0 + \beta_1 x + \epsilon$$ We know that increasing $x$ by 1 increases the log-odds of the event by $\beta_1$, and multiply the odds of the event by $e^{\beta_1}$. But what is the impact of an increase of 1 in $x$ on the probability of the event?

$\endgroup$
2
  • $\begingroup$ @frank so that change cannot be expressed in terms of a single addition or a single multiplication, right? For instance, I can say that a change of 1 in x adds $\beta_1$ to the log-odds and multiply the odds by $e^{\beta_1}$. I cannot say something "simple" like that for probablities right? $\endgroup$ May 28, 2022 at 5:14
  • $\begingroup$ I have written an answer. $\endgroup$
    – frank
    May 28, 2022 at 7:23

1 Answer 1

2
$\begingroup$

As you pointed out, if the model computes the log-odds from $x$ by $$z(x) = \beta_1x + \beta_0,$$ then, if you increase $x$ by one, the log-odds $z(x)$ are increased by $\beta_1$.

Next, the inverse of the logit function is the logistic sigmoid $\sigma(z)$, so the change $\Delta_1 p$ in probability, when increasing $x$ by one, is $$ \Delta_1 p = \sigma(z(x) + \beta_1) - \sigma(z(x)). $$

This difference depends not only on $\beta_1$ but also on $x$, and cannot be simplified.

But what you probably want is the effect of $x$ on $p$, i.e. a factor that you can multiply small values of $\Delta x$ with to obtain $\Delta p$. But since $\sigma(z)$ is nonlinear, you can only compute linear approximations. To do this, first note that the derivative of $\sigma(z)$ is given by: $$ \sigma^\prime(z) = \sigma(z)(1-\sigma(z)). $$ Then, the following approximation makes sense if $\beta_1$ is small or if $|z| \gg 1$: $$ \begin{align} \frac{\Delta p(x)}{\Delta x} &\approx \sigma^\prime(z(x))\;\beta_1\\ & = \sigma(z(x))\;(1-\sigma(z(x)))\;\beta_1\\ &= \sigma(\beta_1x + \beta_0)\;(1-\sigma(\beta_1x + \beta_0))\;\beta_1. \end{align} $$

Thus, in summary: For small $\Delta x$, small $\beta_1$, or $|z| \gg 1$, an approximation of the effect $\Delta p/\Delta x$ is $\sigma(\beta_1x + \beta_0)\;(1-\sigma(\beta_1x + \beta_0))\;\beta_1$, but this varies with $x$.

$\endgroup$
2
  • $\begingroup$ Shouldn't it be $\Delta p(x) = \dfrac{\Delta p(x)}{\Delta x} \Delta x = \sigma'(z(x)) \cdot 1$? Because the increment in $x$ is 1, not $\beta_1$ $\endgroup$ May 28, 2022 at 10:02
  • $\begingroup$ @robertspierre This equation is more general, where the increment in $x$ is $\Delta x$. And if $\Delta x = 1$, we have approximately $\Delta p(x) \approx \sigma^\prime(z(x))\beta_1$. $\endgroup$
    – frank
    May 28, 2022 at 11:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.