Does the sum of updated mixture weights in EM algorithm equal 1 in M step, why? I am usign EM algorithm to estimate my model's parameters. As you know, the mixture weight must be sum to one. $\sum \pi_n = 1$ where $n$ is the number of mixture component. In M step we can find the updated mixture weight for each iteration as follows: $\pi^{i+1} = mean(\tau)$ where $i$ is the number of iteration. My question here, how to prove that, the sum of updated mixture weight will also equal to one? 
Any help please?
 A: Let us say that we consider a mixture model of $K$ components and the inner density is $f(\dot;\Theta)$, i.e. a gaussian density if we consider GMMs and so on.
First of all the Q-function looks like
$$Q(\Theta, \Theta^{(\text{old})}) = \sum_{i=1}^n \sum_{k=1}^K T_{i,k}^{(\text{old})} \left[ \log \tau_k + \log f(x_i;\Theta)\right]$$
(see https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm for the GMM case with $k=2$ but it works just as fine with any other density $f(\dot;\Theta)$ and $K$ arbitrary).
It is easy to see that whenever you have two functions $F, G$ then
  $$ \text{argmax}_{\Theta} (F(\Theta) + G(\Theta)) = \text{argmax}_{\Theta} F(\Theta) + \text{argmax}_{\Theta} G(\Theta)$$
so that we can maximize the $\tau$ and the $\Theta$ individually. Now we want to get
$$\text{argmax}_{\tau} \sum_{k=1}^K \left(\underbrace{\sum_{i=1}^n T_{i,k}^{(\text{old})}}_{=: A_k} \right) \log(\tau_k)$$
under the condition that $\sum_{k=1}^K \tau_k = 1$ and $0 < \tau_k < 1$, i.e. the fact that $\sum_k \tau_k = 1$ does not pop out magically but rather we demand it and we maintain it throughout the execution of the EM algorithm and we have to work for it.
Put $f(\tau) = \sum_{k=1}^K A_k \log(\tau_k)$. Now, solving this problem is just the right application for the theorem of Lagrange (https://en.wikipedia.org/wiki/Lagrange_multiplier), i.e. we put
$$g(\tau) = (\sum_{k=1}^K \tau_k) - 1$$
and now we know that any maximal $\tau$ must be a 'critical' point of the so-called Lagrange function
$$L(\tau, \lambda) = f(\tau) - \lambda g(\tau)$$
i.e. we differentiate by each $\tau_k$ and by $\lambda$ and all these have to be zero. Differentiating by $\tau_k$ (and setting to zero) yields
$$0 = \partial_{\tau_k} L = \frac{A_k}{\tau_k} - \lambda$$
i.e.
$$\tau_k = -\frac{A_k}{\lambda}$$
differentiating by $\lambda$ and setting to zero yields the (original) restriction
$$0 = \partial_{\lambda} L = g(\tau) = (\sum_{k=1}^K \tau_k) - 1$$
so that
$$ 1 = \sum_{k=1}^K \tau_k = -\sum_{k=1}^K \frac{A_k}{\lambda} = \frac{-\sum_{k=1}^K A_k}{\lambda}$$
thus
$$\lambda = -\sum_{k=1}^K A_k$$
and
$$\tau_k = -\frac{A_k}{\lambda} = \frac{A_k}{\sum_{k'=1}^K A_{k'}}$$
Now it is easy to see that $\sum_{k} \tau_k = 1$ but this is not a suprise!!! We created the $\tau_k$ in a way that this equation must be satisfied.
Long answer cut short:
We do not prove it, we maintain this property throughout.
Regards,
FW
