2
$\begingroup$

I use finite mixture models. For two sets $A$ and $B$ of starting values (the initial guess) I get the following values of $Q(\bf{\hat{\Theta}})$ function and $\log L(\bf{\hat{\Theta}})$:

Set $A$: $Q(\bf{\hat{\Theta}_A})=110$ and $\log L(\bf{\hat{\Theta}_A})=7$

Set $b$: $Q({\bf{\hat\Theta}_B})=78$ and $\log L(\bf{\hat{\Theta}_B})=20$

What makes me confused is that for these local probability maxima I should choose $\bf{\hat{\Theta}_A}$ when referring to $Q$ function and conversely choose $\bf{\hat{\Theta}_B}$ if I decided to rely on $\log L$. McLachlan on page 78 (The EM Algorithm and Extensions - McLachlan, Krishnan 2ed. (2008)) allows me to derive that if $\bf{\hat{\Theta}_B}$ doesn’t globally maximizes likelihood function, there exists $\bf{\hat{\Theta}_A}$ that must satisfy: $Q(\bf{\hat{\Theta}_B}) < Q(\bf{\hat{\Theta}_A})$ (which is true for my case) implying that $\log L(\bf{\hat{\Theta}_B}) < \log L(\bf{\hat{\Theta}_A})$. The last inequality doesn't hold for the above values. It means that based on theoretical grounds that something is wrong with my implemented algorithm. Do you think that above derivations are appropriate?

$\endgroup$
4
  • $\begingroup$ I am hoping I can help you with this question but two things are very unclear to me. What do you mean by incomplete? Are you using the EM algorithm because you need to handle a missing data problem? By starting values are you referring to the initial guess at the parameter values? If so it is possible that the likelihood function has more than one local maximum and the starting values in A are far enough apart from those in B that they take you to different solutions. Also if you have missing data a missingness assumption has implications on how to solve the problem. $\endgroup$ Sep 20, 2012 at 17:41
  • $\begingroup$ Maybe multiple imputation should be applied. But I am getting ahead of myself because i don't really know yet what incomplete means when you compute logLc. $\endgroup$ Sep 20, 2012 at 17:42
  • $\begingroup$ I use finite mixture distributions (FMD), and referring to starting values I meant the initial guess. In FMD standard MLE is being from data (x_1,x_2,...,x_n) that can be viewed as being incomplete. Thus, we introduce as the unobservable or missing data the vector z=(z1,z2,...,z_n). Then we have logLc(param|x,z) but logL(param|x). Let's assume that I have 2 local maxima. Based on logLc I have to choose first but using logL my decision is different. Does a theory say that it's possible? $\endgroup$
    – Robert
    Sep 20, 2012 at 18:49
  • $\begingroup$ Sorry for confusion but actually I should have written about $Q$-function instead of logLc. Because I compare $Q$-function=E[loglc|x] and logL. $\endgroup$
    – Robert
    Sep 21, 2012 at 11:38

1 Answer 1

1
$\begingroup$

I'm not sure of the differences you are referring to in logL and logLC, but since they are converging to different values you have the well known problem of converging to a local maxima and not the global maxima (common on EM settings). You should try many different starting values/and or different variants of EM if local maxima is a problem.

The turboEM package in R is a handy tool to experiment with variants of the EM algorithm.

$\endgroup$
2
  • $\begingroup$ Yes, local maxima is a problem but the question is whether it cause the problem I describe. I can't use written procedure in R (not a standard model) but thanks for the package. $\endgroup$
    – Robert
    Sep 20, 2012 at 19:03
  • $\begingroup$ I would look closely at the package, it is very versatile. $\endgroup$
    – Glen
    Sep 20, 2012 at 20:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.