# Why probit regression is less interpretable than logistic regression?

What I've seen very often, is that people are saying, that logistic and probit regression are giving very similar results, however, logistic regression is more interpretabe. And in this question, I want to understand why.

In the logistic regression, interpretability lies on the log of the odds:

$$\log\left(\frac{p}{1 - p}\right) = \beta_0 + \sum_{i = 1}^n \beta_iX_i,$$

because if we take the derivative with respect to $$j-$$ th variable, we exactly obtain the $$j-$$th coefficient:

$$\frac{\partial\log\left(\frac{p}{1 - p}\right)}{\partial X_j} = \beta_j.$$

Because this happens, people are saying, that increase $$X_j$$ of one unit, increase the odds (i.e. $$\frac{p}{1-p}$$) of $$\exp(\beta_j)$$ units.

In the probit regression we can come up with the exactly same thing. We have the following relationship:

$$p = \Phi\left(\beta_0 + \sum_{i = 1}^n \beta_i X_i\right),$$

where $$\Phi$$ is the PDF of standard normal distribution. From this equation we have that:

$$Q(p) = \beta_0 + \sum_{i = 1}^n \beta_i X_i,$$

where $$Q$$ is a quantile function of standard normal distribution.

Now if we take the derivative :

$$\frac{\partial Q(p)}{\partial X_j} = \beta_j$$

So can we say the same? What I would like to say, is that increase $$X_j$$ by one unit, increases quantile function by one unit, and therefore, increase $$X_j$$ by one unit, increases $$p$$ by $$\Phi(\beta_j)$$ units. If that's true, why people are saying, that probit regression is less interpretable?

• Perhaps because the ratio of two odds --- the "odd ratio" --- is a concept well known beyond logistic regression? I've never seen anyone using the ratio of two quantiles, much less heard a name for it. Commented Sep 7, 2022 at 11:21
• Having to take a derivative to make something interpretable is not what I'd recommend. Commented Sep 7, 2022 at 11:55
• "increases $p$ by $\Phi(β_j)$ units" looks peculiar - if $\beta_j=1$, are you saying an increase of $X_j$ by one unit increases $p$ by $0.841$? You would not want to do that twice. Or are you saying it increases $p$ by $\Phi(\hat Y +\beta_j)-\Phi(\hat Y)$ perhaps approximately $\beta_j \phi(\hat Y)$ - neither of which seem easy to understand Commented Sep 7, 2022 at 11:58
• Count me as one of them, in terms of coefficient interpretation. Commented Sep 7, 2022 at 13:06
• But the function $\Phi(x)$ is very interpretable not? A change of $x$ is a change of one standard variation on a normal bell curve. It is maybe not as easy to compute the exact probability as with log-odds, but it is not difficult to interpret. There are even fields where they describe probabilities in terms of the number of $\sigma$. Commented Sep 7, 2022 at 13:09

Logistic regression is a model for probabilities of binary events. Another concept that is closely related to probabilities is odds, i.e. ratios of probabilities. If the probability of observing a binary event is $$p$$, the odds of observing it is $$\tfrac{p}{1-p}$$. It's a fairly simple and commonly understood concept. Logistic regression predicts log-odds. They are "simpler" to interpret because odds are already related to probabilities of binary events, while normal quantiles do not directly translate to them in a meaningful way. If the predicted quantile is $$Q(p)$$, how "likely" is this to happen? To answer the question, you need to translate the value to probability, while in the case of odds it just re-phrases the question to probability relative to the probability of the opposite event.
Let $$\theta = β_0 +\sum_{i=1}^n \beta_i X_i$$ be your linear predictor. If $$X_j$$ increases by one unit, the linear predictor increases by $$\beta_j$$. What can we say about the corresponding change in probability, $$|p(\theta+ \beta_j)-p(\theta)|$$?
In a logistic regression model, the probability is given by $$p(\theta)=e^\theta/(1+e^\theta)$$, the inverse of the logit (log-odds) function. It is straightforward to show that the derivative of this function is maximised at $$\theta=0$$, and that $$p'(0)=1/4$$. We deduce that $$|p(\theta+ \beta_j)-p(\theta)| < \frac{|\beta_j|}{4}$$. In other words, the maximum effect a unit change in the $$j$$th variable can have on the probability is equal to its coefficient divided by 4.