Obviously in linear regression, the coefficient tells you whether the effect of a change in an explanatory variable on the response variable is positive or negative and how much a change of one unit of the explanatory variable effects the reponse variable.

For a binary probit model, I understand you can't interpret the coefficients in the same way. The sign of the coefficient will tell you whether, everything else held constant, a change in a particular explanatory variable has a negative or positive affect on the probability of a response variable being true, but I'm not sure what else can be interpreted apart from that.

I have fitted a binary probit model to a dataset using mainly dummy variables as explanatory variables, and other than saying whether each has a positive/negative effect on the response and the percentage of correctly classified observations the model has when run on the dataset, I'm not sure what else I can say about it. Any ideas?

  • $\begingroup$ To clarify, what type of variable is your response? Why did you choose the binary probit model? $\endgroup$ – dankernler May 12 '18 at 18:00
  • $\begingroup$ The response is a binary variable codes 1 for if an individual is employed and 0 if unemployed/not participating in labour force. Is there a more appropriate model? $\endgroup$ – Wolff May 12 '18 at 22:08
  • $\begingroup$ That sounds to me like binary logistic regression with the logit model. The outcome is the log odds of being employed. I'll explain in an answer. $\endgroup$ – dankernler May 12 '18 at 22:15

Actually, the interpretation from the linear regression framework does indeed apply for the binary profit model. The only problem is, you are now talking about a change of one unit of the explanatory variable effecting a variable hidden behind a curtain that in turn affects the response variable of interest. The challenge in interpreting probit models is in translating this for a general audience.

The model predicts an intermediary latent value, say $y^*$. This value changes in the exact same fashion as in any linear regression framework (e.g., a dichotomous independent variable captures the difference between prediction for two groups, or a 1 unit change in $x$ results in a $b$-unit change in $y^*$). But, the $y^*$ then needs to be "transformed" back to a probability. And this is where the probit transform is utilized. The probability of success is the probability of obtaining $z$-score less than $y^*$.

Thus, the $y^*$ (behind the curtain) is changing in a linear manner, but the final predicted probability is not. And, the same unit "change" can result in very different changes in probabilities. To demonstrate this, take an overly simplified model: $$y^* = -18 + 4x$$ If $x=0$, $y^* = -18$. If $x=1$, $y^* = -14$. Same change in $y^*$...but the change in the predicted probability of success if negligible (compare $P(z \le -18)$ and $P(x \le -14)$...$x=1$ has higher chance of success, but both are just short of impossible). Now, if $x=4$, $y^*=-2$, $P(z \le -2) = 2.3\%$, and $x=5$, $y^*=2$, $P(z \le 2) = 97.7\%$. This suggests a transition from almost definitely not happening to just about surely happening. If you take it to the next extreme, using say $x=10$ and $x=11$, you have two probabilities suggesting the event will definitely happen.

So, the effect on $y^*$ can be interpreted the same as any linear regression...the challenge is in translating this to a probability for $y$ happening.

| cite | improve this answer | |
  • 1
    $\begingroup$ That's helpful. I forgot about the latent variable. However, I'm still unsure about interpreting the coefficients to make general statements. Say I'm comparing labour participation for two separate years where the binary variable "female" (coded 1 for female, 0 for male) is an explanatory variable. If the same explanatory variables are used for separate models on both years, can the coefficients be compared? i.e. If for the first year the coefficient is -2, and the second year it is -1. Can you conclude that the effect on the probability has reduced since the first year? $\endgroup$ – Wolff May 12 '18 at 17:58

Because your outcome is binary, this sounds like a binomial logistic regression (logit model) would be more appropriate. In this case, the coefficients of each of your dummy variables are the change in the log odds of being employed for that predictor from its reference category.

If you exponentiate those, you get an odds ratio compared to the reference category, where all other predictors are held at 0.

The general model is $$\ln{\frac{p}{1-p}}=\beta_0+\beta_1X_1+...\beta_kX_k$$

If we exponentiate both sides, we get $$\frac{p}{1-p}=e^{\beta_0+\beta_1X_1+...\beta_kX_k}$$

So each unit increase in $X_j$ increases the odds by a factor of $e^{\beta_j}$. Since your predictors are dummy variables, a "unit increase" is really just saying what the odds are compared to the reference category.

Note that this is a multiplicative factor, so your wording is "the odds of being employed for [category $j$] are $e^{\beta_j}$ times those of [reference category]".

Further reference is in the Wikipedia page

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.