non-EM algorithm approach to mixture model? I have a mixture model and the components are further parameterized by ~200 variables. Originally I use EM-algorithm to get a MLE estimation of the parameters. The algorithm works quite well and converges quickly.
However, when I scale up the problem to ~400 variables, the problem become ill-conditioned. After some regularization, I have already overcome the numerical difficulty and I could get a meaningful answer, but the computation time of EM algorithm is too long for practical use. I am currently thinking whether I should switch to gradient-based method. But the gradient of the log-likelihood is very complicated (and may be numerical unstable too) if I do not employ EM-algorithm.
May I have some advice on what I algorithm or method I should look for?
P.S. I don't want to use MCMC / Fully Bayesian approach to the problem as I want a MLE estimation.
 A: I suggest you look more carefully at the model before discarding EM. Here are a few points which I hope will help. The most important point is : 
1) Ill conditioned result does not suggest EM has messed up. It suggests the problem setup may not be most appropriate. Try making the model simpler. One step where I generally notice scope for improvement is the covariance matrix. Consider making it diagonal to avoid overfitting.
2) It is completely NOT true that EM will not work when you increase the dimension size. On the contrary, if you look up the Probabilistic PCA (Tipping and Bishop), you will notice that as dimension size reduces, most authors recommend the EM algorithm despite the existence of an analytic solution!!! The convergence is much faster.
3) It must be noted that Variational Bayesian approaches like MCMC are extensions of EM so I wouldn't expect a different solution.
ILL CONDITIONING HAS TO DO WITH THE NATURE OF THE RESULT. IT IS NOT THE ALGORITHM'S FAULT THAT YOUR SOLUTION IS ILL CONDITIONED. YOU NEED TO MODIFY THE SETUP TO IMPROVE THE CONDITION NUMBER OF YOUR OUTPUT
