0
$\begingroup$

I guess this is more a math question than a statistics question. I do not understand how the value of a first derivative can be larger than the range of the original function. I must have a fundamental mistake but I fail to locate it.

The logistic CDF is $$\Lambda(X\beta) \in [0,1]$$ and in this case $$X\beta = \beta_1 x_1 + \beta_2 x_2 + \beta_3 x_1^2x_2$$

Then the marginal effects of $x_1$ and $x_2$ are

$\frac{\partial \Lambda(X\beta)}{\partial x_1} = (\beta_1 + 2\beta_3x_1x_2)\lambda(X\beta)$

$\frac{\partial \Lambda(X\beta)}{\partial x_2} = (\beta_2 + \beta_3x_1^2)\lambda(X\beta)$

The R example below shows that there are many (82 out of 1000) cases where the first derivatives are larger than $\pm1$

set.seed(123);library(data.table)

n = 1000
b1 = .72
b2 = -.86 
b3 = 2.91

x1 <- rnorm(n = n, sd = 1)
x2 <- rnorm(n = n, sd = 1)

xb = (b1 * x1) + (b2 * x2) + (b3 * x1^2 * x2)

dx1 = (b1 + 2 * b3 * x1 * x2) * dlogis(xb)
dx2 = (b2 + b3 * x1^2) * dlogis(xb)

res <- data.table(x1, x2, dx1, dx2)
head(res[abs(dx1)>=1 | abs(dx2)>=1])

           x1          x2        dx1         dx2
1:  1.5587083 -0.01798024  0.1089530  1.21497093
2:  0.3598138  2.83222602  1.2401926 -0.09011081
3: -0.6250393 -1.64849482  1.3765467  0.05674045
4: -1.6866933  0.22855697 -0.3596850  1.75134047
5:  2.1689560 -0.26083224 -0.3165865  1.57885530
6:  0.6443765 -1.69186241 -1.4007296  0.08673243
$\endgroup$
0
$\begingroup$

Just because the values of a function are bounded, does not mean that its derivatives are bounded. Think about the function

$$ f(x)=\sin(1/x) $$

Clearly bounded, isin't it? But guess what happens when you do $x\to 0$? The local variations and therefore the gradient will explode.

In fact, in the logistic regression there is no restriction on the parameter values $\beta_i$ so why should the expression $(a+\beta_i*x_j)*boundedterm$ be bounded?

The left-side multiplier can be anything.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.