in andrew ng lectures notes for expectation maximization, i believe the only assumption invoked for the convergence of the EM algorithm is the jensen inequality that operates on the function Log(x), which is strictly concave over its domain. E[Log(x)] >= Log(E[x])

does this mean that even if the initial parameter guess is in the non-concave part (but still in the positive gradient area) of the local maxima of the log likelihood, the EM algorithm will still find this local maxima since the jensen assumption only operates on the Log function (not on the log likelihood)? or did i misunderstand andrew ng's notes?

my real question is: does this mean that the basin of convergence of a local maxima between the EM and gradient ascent is the same size?

the reason i ask is that i don't know where my initial parameter guesses will be in the log likelihood landscape. will it be really close to the MLE, or sort of offset by some noise so local concavity cannot be guaranteed? to ameliorate these concerns, it would be nice to invoke an interative maximization algorithm that is equipped with minimal assumptions for convergence on a local maximum.

  • $\begingroup$ oops.. let me rephrase that EM converges not to a local extrema, but to zero gradient...but my question about the size of the basin of convergence between EM and Gradient Ascent stil stands $\endgroup$ – hellopluto Jul 15 '15 at 14:54
  • $\begingroup$ "in andrew ng lectures notes for expectation maximization" -> link? $\endgroup$ – Franck Dernoncourt Oct 12 '15 at 5:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.