Does anybody have any source containing explanation of steps in estimating coefficients of multinomial probit model (from likelihood function to first and second derivatives)? Thanks in advance.


The dependent variable follows a multinomial distribution with J levels:

$f(y|\beta)=\prod_{i=1}^N\left\{\frac{n_i!}{\prod_{j=1}^Jy_{ij}!}\cdot \prod_{j=1}^J\pi_{ij}(\beta)^{y_{ij}}\right\}$

Since we want to maximize this with respect ot $\beta$, the factorial term that do not contain any of the $\pi_{ij}$ terms can be treated as constant. Thus, the first step would be specifying the correct form of the multinomial probit likelihood function:


or if we take the natural logarithm:

$ln \ell=\sum_{i=1}^N\sum_{j=1}^{J-1} y_{ij} \cdot ln \Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})+ ({n_i-\sum_{j=1}^{J-1}y_{ij}}) \cdot ln[1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})]$

Would this be the correct form of the kernel of multinomial probit likelihood function which we need to maximize? Is there any simpler form?

How to proceed with the second and third step - first and second derivatives of the log-likelihood function? I would prefer if someone could help a self-learner with the symbolic computation (not in the matrix form).

EDIT #2 (second derivative of the log-likelihood function based on Xi'an's answer):

$\dfrac{\partial^2}{\partial\beta_{kj}\beta'_{kj}}=-\sum_{i=1}^N \cdot\left\{ y_{ij} \cdot \dfrac{(\sum_{k=1}^Kx_{ik}\beta_{jk})\cdot\phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}{\Phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}+y_{ij}\cdot\frac{\phi^2(\sum_{k=1}^Kx_{ik}\beta_{jk})}{\Phi^2(\sum_{k=1}^Kx_{ik}\beta_{jk})}-y_{iJ}\cdot\frac{(\sum_{k=1}^Kx_{ik}\beta_{jk})\phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}{[1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{jk})]^2}+y_{iJ}\cdot\frac{(\sum_{k=1}^Kx_{ik}\beta_{jk})\sum_{j=1}^{J-1}(\Phi\sum_{k=1}^Kx_{ik}\beta_{jk})\phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}{[1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{jk})]^2}+y_{iJ}\cdot\frac{[\phi(\sum_{k=1}^Kx_{ik}\beta_{jk})]^2}{{[1-\sum_{j=1}^{J-1}(\Phi\sum_{k=1}^Kx_{ik}\beta_{jk}}]^2} \right\}x_{ik}x'_{ik}$


1 Answer 1


While you can indeed write the derivatives of the above [corrected] log-likelihood $$\dfrac{\partial}{\partial\beta_{kj}}\sum_{i=1}^N \left\{y_{ij} \cdot ln \Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})+ y_{iJ} \cdot ln\left[1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})\right]\right\}$$ as $$\sum_{i=1}^N \left\{y_{ij} \dfrac{x_{ik}\varphi(\sum_{k=1}^Kx_{ik}\beta_{kj})}{\Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})}- y_{iJ} \dfrac{x_{ik}\varphi(\sum_{k=1}^Kx_{ik}\beta_{kj})}{1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})}\right\}$$ I would suggest using instead an EM algorithm for deriving those estimators.

  • $\begingroup$ Could you please see my second edit of original question to check if the second derivative of the log-likelihood function, based on your answer, is correct? It would be very much appreciated. $\endgroup$
    – Quirik
    Commented Sep 27, 2016 at 8:39
  • $\begingroup$ I am not sure if I understand the first part of your comment so if you could explain it in a little more detail. With respect to the second part, the second derivative of the first fraction in the brackets would then be: $-x'_{ik}\dfrac{\phi^2(\cdot)}{\Phi^2(\cdot)}-x'_{ik}(x_{ik}\beta_{jk})\dfrac{\phi(\cdot)}{\Phi(\cdot)}$ ? $\endgroup$
    – Quirik
    Commented Sep 27, 2016 at 9:04
  • $\begingroup$ When you say the derivative is incorrect, are you reffering now to the whole derivative or just the derivative of the first fraction? $\endgroup$
    – Quirik
    Commented Sep 27, 2016 at 10:08
  • $\begingroup$ For the first fraction: $-x'_{ik}x_{ik}\cdot\dfrac{(\sum_{k=1}^Kx_{ik}\beta_{jk})\cdot\phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}{\Phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}-x'_{ik}x_{ik}\cdot\frac{\phi^2(\sum_{k=1}^Kx_{ik}\beta_{jk})}{\Phi^2(\sum_{k=1}^Kx_{ik}\beta_{jk})}$ ? $\endgroup$
    – Quirik
    Commented Sep 27, 2016 at 10:26
  • $\begingroup$ I meant $-x_{ik'}$ I guess this would be if $j'=j$ if I understood you correctly about taking derivative wrt arbitrary pair. $\endgroup$
    – Quirik
    Commented Sep 27, 2016 at 10:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.