# Does EM algorithm consistently estimate the parameters in Gaussian Mixture model?

I am studying the Gaussian Mixture model and come up with this question myself.

Suppose the underlying data is generated from a mixture of $K$ Gaussian distribution and each of them has a mean vector $\mu_k\in\mathbb{R}^p$, where $1\leq k\leq K$ and each of them has the same co-variance matrix $\Sigma$ and assume this $\Sigma$ is a diagonal matrix. And assume the mixing ratio is $1/K$, i.e., each cluster has same weight.

So in this ideal example, the only job is to estimate the $K$ mean vectors $\mu_k\in\mathbb{R}^p$, where $1\leq k\leq K$ and the co-variance matrix $\Sigma$.

My question is: if we use EM algorithm, will we be able to consistently estimate $\mu_k$ and $\Sigma$, i.e., when sample size $n\rightarrow\infty$, will the estimator produced by EM algorithm achieve the true value of $\mu_k$ and $\Sigma$?

If the algorithm is initialized with random values each time, then no, the convergence will not necessarily be consistent. Non-random initialization will presumably produce the same result every time, but I don't believe that this would necessary produce the "correct" values of $\mu_k$.

As an aside, by fixing the mixing ratio to $1/K$ and fixing $\Sigma$ to be diagonal, the algorithm becomes very similar to the $k$-means algorithm. This also has inconsistent convergence, depending on the random initialization.

• I numerically experimented, at least for 2 independent classes of normal distribution, the EM produces consistent estimator of the class mean. However, K means cannot do that, I proved it mathematically – KevinKim Sep 13 '15 at 2:39
• Could you give more details please? E.g. what data you were using, how you initialised the parameters etc. – dcorney Sep 14 '15 at 8:58
• Agree with @dcorney. It really depends on the initial values you will choose. At least in practice wrong choice of initial values lead to unconsistent estimation (I use mixtools R package) – German Demidov Sep 8 '16 at 9:08