4
$\begingroup$

Does anybody have any source containing explanation of steps in estimating coefficients of multinomial probit model (from likelihood function to first and second derivatives)? Thanks in advance.

EDIT:

The dependent variable follows a multinomial distribution with J levels:

$f(y|\beta)=\prod_{i=1}^N\left\{\frac{n_i!}{\prod_{j=1}^Jy_{ij}!}\cdot \prod_{j=1}^J\pi_{ij}(\beta)^{y_{ij}}\right\}$

Since we want to maximize this with respect ot $\beta$, the factorial term that do not contain any of the $\pi_{ij}$ terms can be treated as constant. Thus, the first step would be specifying the correct form of the multinomial probit likelihood function:

$\ell=\prod_{i=1}^N\prod_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})^{y_{ij}}\cdot[1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})]^{n_i-\sum_{j=1}^{J-1}y_{ij}}$

or if we take the natural logarithm:

$ln \ell=\sum_{i=1}^N\sum_{j=1}^{J-1} y_{ij} \cdot ln \Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})+ ({n_i-\sum_{j=1}^{J-1}y_{ij}}) \cdot ln[1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})]$

Would this be the correct form of the kernel of multinomial probit likelihood function which we need to maximize? Is there any simpler form?

How to proceed with the second and third step - first and second derivatives of the log-likelihood function? I would prefer if someone could help a self-learner with the symbolic computation (not in the matrix form).


EDIT #2 (second derivative of the log-likelihood function based on Xi'an's answer):

$\dfrac{\partial^2}{\partial\beta_{kj}\beta'_{kj}}=-\sum_{i=1}^N \cdot\left\{ y_{ij} \cdot \dfrac{(\sum_{k=1}^Kx_{ik}\beta_{jk})\cdot\phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}{\Phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}+y_{ij}\cdot\frac{\phi^2(\sum_{k=1}^Kx_{ik}\beta_{jk})}{\Phi^2(\sum_{k=1}^Kx_{ik}\beta_{jk})}-y_{iJ}\cdot\frac{(\sum_{k=1}^Kx_{ik}\beta_{jk})\phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}{[1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{jk})]^2}+y_{iJ}\cdot\frac{(\sum_{k=1}^Kx_{ik}\beta_{jk})\sum_{j=1}^{J-1}(\Phi\sum_{k=1}^Kx_{ik}\beta_{jk})\phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}{[1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{jk})]^2}+y_{iJ}\cdot\frac{[\phi(\sum_{k=1}^Kx_{ik}\beta_{jk})]^2}{{[1-\sum_{j=1}^{J-1}(\Phi\sum_{k=1}^Kx_{ik}\beta_{jk}}]^2} \right\}x_{ik}x'_{ik}$

$\endgroup$
  • 1
    $\begingroup$ Since you have used the self study tag, can you tell us what you have tried so far? What do you think the first step would be? (Probably it would involve writing out the model equation to be estimated.) $\endgroup$ – GeoMatt22 Sep 17 '16 at 17:16
  • $\begingroup$ @GeoMatt22 I have further detailed my question. $\endgroup$ – quirik Sep 18 '16 at 18:47
1
+50
$\begingroup$

While you can indeed write the derivatives of the above [corrected] log-likelihood $$\dfrac{\partial}{\partial\beta_{kj}}\sum_{i=1}^N \left\{y_{ij} \cdot ln \Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})+ y_{iJ} \cdot ln\left[1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})\right]\right\}$$ as $$\sum_{i=1}^N \left\{y_{ij} \dfrac{x_{ik}\varphi(\sum_{k=1}^Kx_{ik}\beta_{kj})}{\Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})}- y_{iJ} \dfrac{x_{ik}\varphi(\sum_{k=1}^Kx_{ik}\beta_{kj})}{1-\sum_{j=1}^{J-1}\Phi(\sum_{k=1}^Kx_{ik}\beta_{kj})}\right\}$$ I would suggest using instead an EM algorithm for deriving those estimators.

$\endgroup$
  • $\begingroup$ Could you please see my second edit of original question to check if the second derivative of the log-likelihood function, based on your answer, is correct? It would be very much appreciated. $\endgroup$ – quirik Sep 27 '16 at 8:39
  • $\begingroup$ I am not sure if I understand the first part of your comment so if you could explain it in a little more detail. With respect to the second part, the second derivative of the first fraction in the brackets would then be: $-x'_{ik}\dfrac{\phi^2(\cdot)}{\Phi^2(\cdot)}-x'_{ik}(x_{ik}\beta_{jk})\dfrac{\phi(\cdot)}{\Phi(\cdot)}$ ? $\endgroup$ – quirik Sep 27 '16 at 9:04
  • $\begingroup$ When you say the derivative is incorrect, are you reffering now to the whole derivative or just the derivative of the first fraction? $\endgroup$ – quirik Sep 27 '16 at 10:08
  • $\begingroup$ For the first fraction: $-x'_{ik}x_{ik}\cdot\dfrac{(\sum_{k=1}^Kx_{ik}\beta_{jk})\cdot\phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}{\Phi(\sum_{k=1}^Kx_{ik}\beta_{jk})}-x'_{ik}x_{ik}\cdot\frac{\phi^2(\sum_{k=1}^Kx_{ik}\beta_{jk})}{\Phi^2(\sum_{k=1}^Kx_{ik}\beta_{jk})}$ ? $\endgroup$ – quirik Sep 27 '16 at 10:26
  • $\begingroup$ I meant $-x_{ik'}$ I guess this would be if $j'=j$ if I understood you correctly about taking derivative wrt arbitrary pair. $\endgroup$ – quirik Sep 27 '16 at 10:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.