I am currently studying TwoStep clustering and one of the distance measures that is the log likelihood distance measure. Mainly this distance measure accepts both categorical and continuous variables. For categorical variables it is assumed that the variables are independent multinomial distributed. Can someone help me derive the log likelihood distance for the multinomial distribution.

For better understanding one may refer to the following:


  • $\begingroup$ What do you mean by "help to derive"? The paper explains it quite thoroughly. SPSS Algorithms doc also has these formulas. If you want the theoretic idea behind the log-likelihood (on which AIC, BIC and the distance are based) - consider what is entropy in normal continuous and in multinomial distributions. Also, search log-likelihood distance on this site. $\endgroup$ – ttnphns Oct 1 '17 at 8:27
  • $\begingroup$ I am trying to understand how he did manage to get the log likelihood of the multinomial distribution as I cannot get it the way he did in the paper and which site are you referring to please? $\endgroup$ – Annalise Azzopardi Oct 1 '17 at 8:29
  • $\begingroup$ Search (wikipedia, elsewhere, and the current site, too) for "entropy of multinomial distribution" $\endgroup$ – ttnphns Oct 1 '17 at 8:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.