Marginal effect of Probit and Logit model

Question

Can anyone explain how to compute the marginal effect of Probit and Logit model in layman's terms?

I am new to statistics and I am confused about these two models.

Answer 1

I think a better way to see the marginal effect of a given variable, say $X_j$, is to produce a scatter plot of the predicted probability on the vertical axis, and to have $X_j$ on the horizontal axis. This is the most "layman" way I can think of indicating how influential a given variable is. No maths, just pictures. If you have a lot of data points, then a boxplot, or scatterplot smoother may help to see where most of the data is (as oppose to just a cloud of points).

Not sure how "Layman" the next section is, but you may find it useful.

If we look at the marginal effect, call it $m_j$, noting that $g(p)=\sum_kX_k\beta_k$, we get

$$m_j=\frac{\partial p}{\partial X_j}=\frac{\beta_j}{g'\left[g^{-1}(X^T\beta)\right]}=\frac{\beta_j}{g'(p)}$$

So the marginal effect depends on the estimated probability and the gradient of the link function in addition to the beta. The dividing by $g'(p)$, comes from the chain rule for differentiation, and the fact that $\frac{\partial g^{-1}(z)}{\partial z}=\frac{1}{g'\left[g^{-1}(z)\right]}$. This can be shown by differentiating both sides of the obviously true equation $z=g\left[g^{-1}(z)\right]$. We also have that $g^{-1}(X^T\beta)=p$ by definition. For a logit model, we have $g(p)=\log(p)-\log(1-p)\implies g'(p)=\frac{1}{p}+\frac{1}{1-p}=\frac{1}{p(1-p)}$, and the marginal effect is:

$$m_j^{logit}=\beta_jp(1-p)$$

What does this mean? well $p(1-p)$ is zero at $p=0$ and at $p=1$, and it reaches its maximum value of $0.25$ at $p=0.5$. So the marginal effect is greatest when the probability is near $0.5$, and smallest when $p$ is near $0$ or near $1$. However, $p(1-p)$ still depends on $X_j$, so the marginal effects are complicated. In fact, because it depends on $p$, you will get a different marginal effect for different $X_k,\;k\neq j$ values. Possibly one good reason to just do that simple scatter plot - don't need to chose which values of the covariates to use.

For a probit model, we have $g(p)=\Phi^{-1}(p)\implies g'(p)=\frac{1}{\phi\left[\Phi^{-1}(p)\right]}$ where $\Phi(.)$ is standard normal CDF, and $\phi(.)$ is standard normal pdf. So we get:

$$m_j^{probit}=\beta_j\phi\left[\Phi^{-1}(p)\right]$$

Note that this has most of the properties that the $m_j^{logit}$ marginal effect I discussed earlier, and is equally true of any link function which is symmetric about $0.5$ (and sane, of course, e.g. $g(p)=tan(\frac{\pi}{2}[2p-1])$). The dependence on $p$ is more complicated, but still has the general "hump" shape (highest point at $0.5$, lowest at $0$ and $1$). The link function will change the size of the maximum height (e.g. probit maximum is $\frac{1}{\sqrt{2\pi}}\approx 0.4$, logit is $0.25$), and how quickly the marginal effect is tapered towards zero.

Answer 2

The logit and probit models are typically used to figure out a probability that the dependent variable y is 0 or 1 based on a number of input variables.

In English: Suppose you're trying to predict a binary value, such as whether or not somebody will develop heart disease during their life. You have a number of input variables such as blood pressure, age, whether or not they are a smoker, their BMI, where they live, etc. etc. All those variables may contribute in some way to the chances of somebody developing heart disease.

The marginal effect of a single input variable is if you raise that variable by a bit, how does that affect the probability of having heart disease? Suppose blood pressure increases by a slight amount, how does that change the chances of having heart disease? Or if you raise the age by a year?

Some of these effects could also be non-linear: increasing BMI by a slight amount may have a very different effect for somebody who has a very healthy BMI than for somebody who does not.

Answer 3

You'd still want your layman to know the calculus, as marginal effect is the derivative of a fitted probability with respect to the variable of interest. As fitted probability is the link function (logit, probit or whatever) applied to the fitted values, you need the chain rule to compute it. So, in linear index models (where parameters enter as something like X'b) it is equal to the parameter estimate times the derivative of the link function. As the derivative is different at different values of the regressors (unlike the case of a linear model), you have to decide, where to evaluate the marginal effect. A natural choice would be mean values of all the regressors. Another approach would be to evaluate the effect for the each observation and then average over them. The interpretation differs accordingly.