4
$\begingroup$

Similarly to the question Difference between logit and probit models I am wondering what is the difference between a multinomial logit and a multinomial probit. And when should I apply which of the two algorithms?

$\endgroup$
  • $\begingroup$ I think you could be mixing up multinomial probit with IIA (which is not alternative specific) and the alternative-specific multinomial probit, which allows for alternative-specific and case-specific independent variables. The alternative-specific data is what identifies the error correlations and allows you to relax the IIA. $\endgroup$ – Dimitriy V. Masterov Sep 23 '16 at 19:09
  • $\begingroup$ I deleted the missunderstanding part of the question. I am only interested in the difference between the two algorithms $\endgroup$ – Ferdi Sep 24 '16 at 13:04
5
$\begingroup$

A very nice, formal explanation of the difference between multinomial logit (with IIA), multinomial probit (with IIA), and alternative-specific multinomial probit (without IIA) is provided by Long and Freese (2014, p.465-479):

Let's think a random utility model, where $u_{im}$ is the utility for person $i$ from alternative $m$, and it is determined by a linear combination of observed characteristics $x_i$ and random error $\varepsilon_{im}$:

$$u_{im}=x_{i}\beta_{m}+\varepsilon_{im}$$

A person chooses alternative $j$ when $u_{ij} > u_{im}$ for all $m\neq j$. The probability of choice for $m$ is

$$\text{Pr}(y_i=m)=\text{Pr}(u_{im} > u_{ij} \text{ for all } j \neq m)$$

The choice is based on the difference in utilities between alternatives. So if we assume three alternatives and taking one of them as base, the equations are

$$u_{i1}-u_{i1}=0$$ $$u_{i2}-u_{i1}=x_i(\beta_2-\beta_1)+(\varepsilon_{i2}-\varepsilon_{i1})$$ $$u_{i3}-u_{i1}=x_i(\beta_3-\beta_1)+(\varepsilon_{i3}-\varepsilon_{i1})$$

And if we define $u^*_{im} \equiv u_{im} - u_{i1}$, $\varepsilon^*_{im} \equiv \varepsilon_{im}-\varepsilon_{i1}$ and $\beta_{m|1} \equiv \beta_m - \beta_1$, the new equations are

$$u^*_{i2} = x_{i}\beta_{2|1}+\varepsilon^*_{i2}$$ $$u^*_{i3} = x_{i}\beta_{3|1}+\varepsilon^*_{i3}$$

Now, so far it is quite straightforward and the specific form of the model depends on the distribution of errors. So, if $\varepsilon$ is assumed to be distributed logistically (here, with mean 0 and variance $\pi^2/6$), we will have multinomial logit model. If $\varepsilon$ is assumed to be normally distributed we have multinomial probit model. It is just like the difference between binary logit and probit models. However, for multinomial probit model, we can also allow the errors to be correlated or not. So, we have covariance matrix of $\varepsilon$'s for three alternatives:

$$\Sigma_{\varepsilon}= \left[ \begin{array}{ccc} \sigma_{11} & & \\ \sigma_{21} & \sigma_{22} & \\ \sigma_{31} & \sigma_{32} & \sigma_{33} \end{array} \right]$$

If we constrain this matrix ($\Sigma_{\varepsilon}=\text{I}$), we will have a unit variance, normal error distribution, but uncorrelated errors, so the model would have IIA (independence of irrelevant alternatives) property. If we do not constrain, we will have alternative-specific multinomial probit model (as Dimitriy V. Masterov mentioned in the comment). Relaxing IIA condition is one of the main reasons why alternative-specific multinomial probit model is prefered over multinomial logit model. However, Long & Freese (2014) informed us that although correlation of errors relaxes IIA, parameters in $\Sigma_{\varepsilon}$ won't be identified if some constraint is not imposed. They discuss how this can be done, but this is beyond the scope of my answer.

So, why should we apply multinomial probit rather than multinomial logit? What is the advantage of relaxing IIA? To answer the question, we should understand IIA axiom. Wikipedia entry on IIA provides a nice summary. To illustrate the issue, blue bus/red bus problem is given as an example (based on McFadden, 1973). So, suppose that we need to choose between two forms of transportation, car and red bus, and suppose that we choose these two options with equal probability, 0.5. If we introduce a blue bus as an additional alternative, under the assumption of IIA, we should have a new probability, 0.33, for each option. However, this is not very intuitive as two of our options (red bus and blue bus) are quite similar. Another, and maybe more realistic, example could be a choice between four alternative modes of travel: plane, train, car, and bus. Now, under IIA we consider these alternatives independent or distinct, but three of these options can be grouped as ground transportation. Thus, if we estimate a model, we might want to have correlated errors. In this and similar cases, alternative-specific multinomial probit model can be preferred.

I really recommend looking at Scott & Freese (2014) and how they explain and discuss the differences between models.


Long, S. J., & Freese, J. (2014). Regression Models for Categorical Dependent Variables Using Stata. Texas: Stata Press.

McFadden, D. (1973). Conditional Logit Analysis of Qualitative Choice Behavior. In Frontiers in Econometrics (pp. 105–142). New York: Academic Press.

$\endgroup$
  • $\begingroup$ Thank you so much. So in order to summarize it in once sentence. You use the multinomial logit when all the independent variables are uncorrelated of each other. $\endgroup$ – Ferdi Sep 26 '16 at 6:49
  • 3
    $\begingroup$ @Ferdi, I am glad I could help. My answer is a little confusing maybe, but we are not talking about independent variables, but error term ($\varepsilon$) in the model. First, if we assume that the error is normally distributed, then we may choose probit model, if not (i.e., logistically distributed) logit model. After this, if we choose probit model, we can have correlated errors. The examples above illustrate why this might be desirable. So, in one sentence: if the distribution of error is logistic (i.e., not normal) and errors are uncorrelated, you can use multinomial logit. $\endgroup$ – T.E.G. - Reinstate Monica Sep 26 '16 at 13:12
  • 1
    $\begingroup$ @Ferdi This website might be helpful too. $\endgroup$ – T.E.G. - Reinstate Monica Sep 26 '16 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.