1
$\begingroup$

Let's say I have the following model:

$$\ln\Big(\frac{\mathbb{P}(Y_i = 1 | X_i)}{\mathbb{P}(Y_i = 0 | X_i)}\Big) = \beta_0 + \beta_1 X_{1,i} + \beta_2 X_{2,i}$$

Let $\rho = \frac{\mathbb{P}(Y_i = 1 | X_i)}{\mathbb{P}(Y_i = 0 | X_i)}$.

Then: $$\frac{\partial \ln\rho}{\partial X_1} = \frac{\partial \ln\rho}{\partial \rho} \frac{\partial \rho}{\partial X_1}= \frac{\partial \rho}{\rho} \frac{1}{\partial X_1} = \beta_1$$ So if $\partial X_1 = 1$ then $\frac{\partial \rho}{\rho} = \beta_1$.

Similarly, let $\ln\rho^0$ increase to $\ln\rho^1$ after the unit increase in $X_1$. If this is so, then: $$\ln\rho^1 - \ln\rho^0 = \beta_1 \\ \implies \frac{\rho^1}{\rho^0} = e^{\beta_1}$$


Question

Shouldn't it always hold that $e^{\beta_1} - 1 = \beta_1$ when $\partial X_1 = 1$?


My estimated model returned $\hat{\beta_1} = 0.0326$ and:

  • $0.0326 \neq \big[ e^{\hat{\beta_1}} - 1 = 0.0332 \big]$

How come these two effects are not the same?

$\endgroup$
4
  • $\begingroup$ By $\delta$, I assume you are referring to differentials? If this is so, then it makes no sense to state $\delta X_1 = 1$. You are confusing differentials with finite differences. $\endgroup$
    – user277126
    Nov 30, 2021 at 5:45
  • $\begingroup$ Yes, I mean the partial derivative of a function with respect to $X_1$ (I added the term $\beta_2 X_2$). $\endgroup$
    – Arturo Sbr
    Nov 30, 2021 at 5:50
  • 1
    $\begingroup$ You do not have the power to set the value of the differential of $X_1$. The interpretation is made like simple linear regression: using a finite difference (1-unit increase). $\endgroup$
    – user277126
    Nov 30, 2021 at 5:54
  • $\begingroup$ You might find the extended discussion at stats.stackexchange.com/questions/133623 to be helpful. $\endgroup$
    – whuber
    Nov 30, 2021 at 17:05

1 Answer 1

1
$\begingroup$

Let $\ln(\rho)=\beta_0+\beta_1x_1+\beta_2x_2$. This is called the log-odds. The partial derivative of $\ln(\rho)$ with respect to $x_1$ is then

$$\frac{\partial}{\partial x_1}\ln(\rho)=\frac{1}{\rho}\cdot\frac{\partial}{\partial x_1}\rho=\frac{\beta_1}{\beta_0+\beta_1x_1+\beta_2x_2}.$$ As correctly noted in comments, there is no way of setting the value of the differential of $x_1$.

Should you wish to assess the effect of a unit increase in $x_1$, the way is using log of the odds ratio: Denote $x,x'$ two vectors where $x'_1=x_1+1$ and the rest of the components unchanged. The log-odds ratio is then:

$$\ln\left(\frac{\rho(x')}{\rho(x)}\right)=\beta_0+\beta_1x'_1+\beta_2x'_2-\beta_0-\beta_1x_1-\beta_2x_2=\beta_1(x_1'-x_1)=\beta_1.$$

Another explanation, a bit more abstract: in logistic regression, much like in linear regression, we compute the linear predictor $\theta_i=x_i^T\beta$. In order to obtain a probability, we put it into the sigmoid function:

$$P(y_i=1|x_i)=\pi_i=sigmoid(\theta_i)=\frac{1}{1+e^{-\theta_i}}=\frac{e^{\theta_i}}{1+e^{\theta_i}}$$

The inverse of the sigmoid function is called the logit function:

$$logit(\pi_i)=\log\left(\frac{\pi_i}{1-\pi_i}\right)=\log\left(\frac{\frac{e^{\theta_i}}{1+e^{\theta_i}}}{\frac{1}{1+e^{\theta_i}}}\right)=\log(e^{\theta_i})=\theta_i$$ If you look closely, $logit(\pi_i)$ is exactly the log-odds for $x_i$, so it all connects.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.