1
$\begingroup$

Having read about logistic regression, I understand that the probabilities of the success of the DV, P(Y=1), do not necessarily grow the same at every level of the independent variable. This is why you can't interpret the coefficients right away like for OLS, saying an increase in one unit of x1 increases P(Y=1) by b1 (=the coefficient of x1). Using margins and marginsplot in Stata, however, can help interpret the results.

However, I'm getting a linear relation in my marginsplot between P(Y=1) and some (maybe every..haven't checked them all) independent variables.

What does this tell me? Is this suggesting, that I'm doing something wrong or indicating that I should look into something again. I'm just curious, because usually you would expect margins to be somehow not linear (that's why you do the marginsplot in the first place)..and this perfect linear relationship kind of sets off my alarm bells :)

$\endgroup$
  • $\begingroup$ are there interactions in your model? I've only used marginsplot in the setting of interactions. Otherwise I think (but not sure) it should be linear. $\endgroup$ – charles Nov 15 '13 at 1:37
5
$\begingroup$

To get a sense of what might be going on here, consider a simple logistic regression model with one continuous regressor

$$P(Y=1\mid X) = \Lambda (g(x)) =\Big(1 + \exp\{g(x)\}\Big)^{-1} =\Big(1 + \exp\{a+bx\}\Big)^{-1}$$

The marginal effect is

$$\frac {\partial P}{\partial X} = \Lambda (1-\Lambda)b $$

The 2nd-order Maclaurin series of $\Lambda$ (its Taylor expansion centered at zero) with respect to the logit is

$$\Lambda \approx \frac 12 +\frac 14 g(x)$$

because the 2nd-order term is zero.

Then

$$\frac {\partial P}{\partial X} \approx \left(\frac 12 +\frac 14 g(x) \right)\left(1-\frac 12 -\frac 14 g(x)\right)b = \left(\frac 14 -\frac 1{16} g(x)^2\right)b$$

$$= \frac 14b -\frac 1 {16}\left(a^2 +2abx + b^2x^2\right)b$$

$$= \frac 14\Big(1 -\frac 1 {4}a^2\Big)b - \frac 18ab^2x - \frac 1{16}b^3x^2$$

So depending on the range of $X$, and the value of $\hat b$, it may be the case that the 2nd order approximation is good and the last term that features the regressor squared is "small", hence giving an approximate linear relation between the probability and the regressor.

$\endgroup$
6
$\begingroup$

To put the argument of @alecos-papadopoulos in graphical terms:

Say his $a$ is -1 and his $b$ is 0.5 and $x$ ranges between 0 and 2. If we were to graph that relationship we could type in Stata:

twoway function Pr = invlogit(-1+0.5*x), range(0 2) lwidth(medthick)

That would result in the following graph:

enter image description here

which looks pretty linear. However, if we were to look at this relationship over a larger range of values for $x$, we would see that it is part of a non-linear relationship:

twoway function Pr = invlogit(-1+0.5*x),           ///
                     range(-10 10) lpattern(solid) ///
                     lcolor(gs12) lwidth(thick) || ///
       function Pr = invlogit(-1+0.5*x),           ///
                     range(0 2) lpattern(solid)    ///
                     lcolor(black) lwidth(thick)   ///
                     legend(off)

enter image description here

As a rule of thumb the relationship between $x$ and the predicted probability is approximately linear between a predicted probability of .2 and .8.

$\endgroup$
  • $\begingroup$ Very illustrative example Maarten, thanks for making my answer more intuitive. $\endgroup$ – Alecos Papadopoulos Nov 15 '13 at 21:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.