Why probit regression is less interpretable than logistic regression? What I've seen very often, is that people are saying, that logistic and probit regression are giving very similar results, however, logistic regression is more interpretabe. And in this question, I want to understand why.
In the logistic regression, interpretability lies on the log of the odds:
$$\log\left(\frac{p}{1 - p}\right) = \beta_0 + \sum_{i = 1}^n \beta_iX_i,$$
because if we take the derivative with respect to $j-$ th variable, we exactly obtain the $j-$th coefficient:
$$\frac{\partial\log\left(\frac{p}{1 - p}\right)}{\partial X_j} = \beta_j.$$
Because this happens, people are saying, that increase $X_j$ of one unit, increase the odds (i.e. $\frac{p}{1-p}$) of $\exp(\beta_j)$ units.
In the probit regression we can come up with the exactly same thing. We have the following relationship:
$$p = \Phi\left(\beta_0 + \sum_{i = 1}^n \beta_i X_i\right),$$
where $\Phi$ is the PDF of standard normal distribution. From this equation we have that:
$$Q(p) = \beta_0 + \sum_{i = 1}^n \beta_i X_i,$$
where $Q$ is a quantile function of standard normal distribution.
Now if we take the derivative :
$$\frac{\partial Q(p)}{\partial X_j} = \beta_j$$
So can we say the same? What I would like to say, is that increase $X_j$ by one unit, increases quantile function by one unit, and therefore, increase $X_j$ by one unit, increases $p$ by $\Phi(\beta_j)$ units. If that's true, why people are saying, that probit regression is less interpretable?
 A: Here's another way to think about this:
Let $\theta = β_0 +\sum_{i=1}^n \beta_i X_i $ be your linear predictor. If $X_j$ increases by one unit, the linear predictor increases by $\beta_j$. What can we say about the corresponding change in probability, $|p(\theta+ \beta_j)-p(\theta)|$?
In a logistic regression model, the probability is given by $p(\theta)=e^\theta/(1+e^\theta)$, the inverse of the logit (log-odds) function. It is straightforward to show that the derivative of this function is maximised at $\theta=0$, and that $p'(0)=1/4$. We deduce that $|p(\theta+ \beta_j)-p(\theta)| < \frac{|\beta_j|}{4}$. In other words, the maximum effect a unit change in the $j$th variable can have on the probability is equal to its coefficient divided by 4.
Can you work out the equivalent for the probit model?
A: Logistic regression is a model for probabilities of binary events. Another concept that is closely related to probabilities is odds, i.e. ratios of probabilities. If the probability of observing a binary event is $p$, the odds of observing it is $\tfrac{p}{1-p}$. It's a fairly simple and commonly understood concept. Logistic regression predicts log-odds. They are "simpler" to interpret because odds are already related to probabilities of binary events, while normal quantiles do not directly translate to them in a meaningful way. If the predicted quantile is $Q(p)$, how "likely" is this to happen? To answer the question, you need to translate the value to probability, while in the case of odds it just re-phrases the question to probability relative to the probability of the opposite event.
