# Interpretation of coefficient in logistic regression

Let's say I have the following model:

$$\ln\Big(\frac{\mathbb{P}(Y_i = 1 | X_i)}{\mathbb{P}(Y_i = 0 | X_i)}\Big) = \beta_0 + \beta_1 X_{1,i} + \beta_2 X_{2,i}$$

Let $$\rho = \frac{\mathbb{P}(Y_i = 1 | X_i)}{\mathbb{P}(Y_i = 0 | X_i)}$$.

Then: $$\frac{\partial \ln\rho}{\partial X_1} = \frac{\partial \ln\rho}{\partial \rho} \frac{\partial \rho}{\partial X_1}= \frac{\partial \rho}{\rho} \frac{1}{\partial X_1} = \beta_1$$ So if $$\partial X_1 = 1$$ then $$\frac{\partial \rho}{\rho} = \beta_1$$.

Similarly, let $$\ln\rho^0$$ increase to $$\ln\rho^1$$ after the unit increase in $$X_1$$. If this is so, then: $$\ln\rho^1 - \ln\rho^0 = \beta_1 \\ \implies \frac{\rho^1}{\rho^0} = e^{\beta_1}$$

# Question

Shouldn't it always hold that $$e^{\beta_1} - 1 = \beta_1$$ when $$\partial X_1 = 1$$?

My estimated model returned $$\hat{\beta_1} = 0.0326$$ and:

• $$0.0326 \neq \big[ e^{\hat{\beta_1}} - 1 = 0.0332 \big]$$

How come these two effects are not the same?

• By $\delta$, I assume you are referring to differentials? If this is so, then it makes no sense to state $\delta X_1 = 1$. You are confusing differentials with finite differences. Nov 30, 2021 at 5:45
• Yes, I mean the partial derivative of a function with respect to $X_1$ (I added the term $\beta_2 X_2$). Nov 30, 2021 at 5:50
• You do not have the power to set the value of the differential of $X_1$. The interpretation is made like simple linear regression: using a finite difference (1-unit increase). Nov 30, 2021 at 5:54
• You might find the extended discussion at stats.stackexchange.com/questions/133623 to be helpful.
– whuber
Nov 30, 2021 at 17:05

Let $$\ln(\rho)=\beta_0+\beta_1x_1+\beta_2x_2$$. This is called the log-odds. The partial derivative of $$\ln(\rho)$$ with respect to $$x_1$$ is then

$$\frac{\partial}{\partial x_1}\ln(\rho)=\frac{1}{\rho}\cdot\frac{\partial}{\partial x_1}\rho=\frac{\beta_1}{\beta_0+\beta_1x_1+\beta_2x_2}.$$ As correctly noted in comments, there is no way of setting the value of the differential of $$x_1$$.

Should you wish to assess the effect of a unit increase in $$x_1$$, the way is using log of the odds ratio: Denote $$x,x'$$ two vectors where $$x'_1=x_1+1$$ and the rest of the components unchanged. The log-odds ratio is then:

$$\ln\left(\frac{\rho(x')}{\rho(x)}\right)=\beta_0+\beta_1x'_1+\beta_2x'_2-\beta_0-\beta_1x_1-\beta_2x_2=\beta_1(x_1'-x_1)=\beta_1.$$

Another explanation, a bit more abstract: in logistic regression, much like in linear regression, we compute the linear predictor $$\theta_i=x_i^T\beta$$. In order to obtain a probability, we put it into the sigmoid function:

$$P(y_i=1|x_i)=\pi_i=sigmoid(\theta_i)=\frac{1}{1+e^{-\theta_i}}=\frac{e^{\theta_i}}{1+e^{\theta_i}}$$

The inverse of the sigmoid function is called the logit function:

$$logit(\pi_i)=\log\left(\frac{\pi_i}{1-\pi_i}\right)=\log\left(\frac{\frac{e^{\theta_i}}{1+e^{\theta_i}}}{\frac{1}{1+e^{\theta_i}}}\right)=\log(e^{\theta_i})=\theta_i$$ If you look closely, $$logit(\pi_i)$$ is exactly the log-odds for $$x_i$$, so it all connects.