0
$\begingroup$

As known, EM algorithm is sensitive to the starting values. One method to select the starting values is to run EM several times using different starting values each time. Then, the select the one that return large log -likelihood function.

My question is, can we use the plot of the log-likelihood function vs iteration as an evidence if the starting values is good choice or we need to try another staring values?

For example, After several tries with starting values, I got this convergence plot of my log-likelihood function.is this still valid convergenceand this

$\endgroup$
  • $\begingroup$ I would say "no", because convergence plots may simply show "how well" you are converging toward a local optimum. Analogously, you may want to look at ploting llik profile: which can however be tedious if your llik is maximized over more than 2 parameters. $\endgroup$ – keepAlive Dec 21 '17 at 13:56
  • $\begingroup$ @Kanak my log like function maximized over 12 parameters. However, Do you think my plot is valid or the convergence is not clear $\endgroup$ – Silver_80 Dec 21 '17 at 13:58
  • 1
    $\begingroup$ It is not clear to me: around the 40th iteration (of your first chart) one sees a jump: what if another jump occurs at the 90th iteration? Also, convergence plots are commonly represented with 0-centered 1-based indexes, see, e.g. this. $\endgroup$ – keepAlive Dec 21 '17 at 14:03
  • $\begingroup$ @Kanak Thank so much for your help. do you think that due to the starting values. $\endgroup$ – Silver_80 Dec 21 '17 at 14:03
  • $\begingroup$ Yes. This is very likely. Depending on how costly it is to perform iterations, a good strategy may be to chose distant initial guesses, iterate a high number of times, and shows that all of those (differently initiated maximizations) converge toward the same optimum. Or even better enter into the evil forest and try something like simulated annealing. $\endgroup$ – keepAlive Dec 21 '17 at 14:07
1
$\begingroup$

I would say "no", because convergence plots may simply show "how well" you are converging toward a local optimum. Analogously, you may want to look at ploting log-likelihood profile: which can however be tedious if your log-likelihood is maximized over more than 2 parameters.

Be that as it may, your convergence is not clear: around the 40th iteration (of your first chart) one sees a jump: what if another jump occurs at the 90th iteration? The two different profiles you get are very likely to be due to your initial guesses. Depending on how costly it is to perform iterations, a good strategy may be to chose distant initial guesses, iterate a high number of times, and shows that all of those (differently initiated maximizations) converge toward the same optimum. Or even better enter into the "evil forest" and try something like simulated annealing.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.