# How to reestimate GMMs in a HMM-GMM

Context: Automatic Speech Recognition

I understand the training of a pure HMM with Baum-Welch:

Expectation step

• compute $\gamma_t(i) = P(q_t=i |O,\lambda)$ //p(passing state $i$ at frame $t$)
• compute $\xi_t(i,j) =P(q_t=i,q_{t+1}=j|O,\lambda)$ //p(transition state $i\rightarrow j$ in frames $t,t+1$)

Maximization step

• e.g. $b_i'(v_k)=\frac{\sum\limits_{t=1}^{T}P(q_t=i|O,\lambda)*\delta(o_t=v_k)}{\sum\limits_{t=1}^{T}P(q_t=i|O,\lambda)}$ //Emission probabilities

But our HMMs models the emission probabilities with GMMs.
The script just says in case of GMM do EM (and with NN do backpropagation)
But I don't see how that should work. EM for GMM needs feature vectors and not the information in with state we probably are right? So what is the new information for the GMMs?

If we want different emission probabilities (GMMs, NNs) for different sates, we have to know the probability of seeing the observations given a hidden state.

I would think of this as allocating each hidden state a weight of being responsible for generating an observation, and weights for each observation sum to 1.

Then based on the weights we can do MLE (EM for GMM, BP for NN) for each state separately to estimate the parameters at the M step for HMM.

For GMM-HMMs normally we don't actually run EM for GMM at every step, instead at the E step we estimate the latent variables for GMM and HMM together by using the "component-state occupation probability" $\gamma_t(i,m)$, which is the probability of occupying mixture component $m$ of sate $i$ at time $t$. Then at the M step the MLE for each Gaussian components can be solved directly.