# Impact of change in independent variable on probability in logistic regression

Take the following logistic regression: $$\log(o) = \beta_0 + \beta_1 x + \epsilon$$ We know that increasing $$x$$ by 1 increases the log-odds of the event by $$\beta_1$$, and multiply the odds of the event by $$e^{\beta_1}$$. But what is the impact of an increase of 1 in $$x$$ on the probability of the event?

• @frank so that change cannot be expressed in terms of a single addition or a single multiplication, right? For instance, I can say that a change of 1 in x adds $\beta_1$ to the log-odds and multiply the odds by $e^{\beta_1}$. I cannot say something "simple" like that for probablities right? May 28, 2022 at 5:14
• I have written an answer. May 28, 2022 at 7:23

## 1 Answer

As you pointed out, if the model computes the log-odds from $$x$$ by $$z(x) = \beta_1x + \beta_0,$$ then, if you increase $$x$$ by one, the log-odds $$z(x)$$ are increased by $$\beta_1$$.

Next, the inverse of the logit function is the logistic sigmoid $$\sigma(z)$$, so the change $$\Delta_1 p$$ in probability, when increasing $$x$$ by one, is $$\Delta_1 p = \sigma(z(x) + \beta_1) - \sigma(z(x)).$$

This difference depends not only on $$\beta_1$$ but also on $$x$$, and cannot be simplified.

But what you probably want is the effect of $$x$$ on $$p$$, i.e. a factor that you can multiply small values of $$\Delta x$$ with to obtain $$\Delta p$$. But since $$\sigma(z)$$ is nonlinear, you can only compute linear approximations. To do this, first note that the derivative of $$\sigma(z)$$ is given by: $$\sigma^\prime(z) = \sigma(z)(1-\sigma(z)).$$ Then, the following approximation makes sense if $$\beta_1$$ is small or if $$|z| \gg 1$$: \begin{align} \frac{\Delta p(x)}{\Delta x} &\approx \sigma^\prime(z(x))\;\beta_1\\ & = \sigma(z(x))\;(1-\sigma(z(x)))\;\beta_1\\ &= \sigma(\beta_1x + \beta_0)\;(1-\sigma(\beta_1x + \beta_0))\;\beta_1. \end{align}

Thus, in summary: For small $$\Delta x$$, small $$\beta_1$$, or $$|z| \gg 1$$, an approximation of the effect $$\Delta p/\Delta x$$ is $$\sigma(\beta_1x + \beta_0)\;(1-\sigma(\beta_1x + \beta_0))\;\beta_1$$, but this varies with $$x$$.

• Shouldn't it be $\Delta p(x) = \dfrac{\Delta p(x)}{\Delta x} \Delta x = \sigma'(z(x)) \cdot 1$? Because the increment in $x$ is 1, not $\beta_1$ May 28, 2022 at 10:02
• @robertspierre This equation is more general, where the increment in $x$ is $\Delta x$. And if $\Delta x = 1$, we have approximately $\Delta p(x) \approx \sigma^\prime(z(x))\beta_1$. May 28, 2022 at 11:40