7
$\begingroup$

I want to build a hidden Markov model (HMM) with continuous observations modeled as Gaussian mixtures (Gaussian mixture model = GMM).

The way I understand the training process is that it should be made in $2$ steps.

1) Train the GMM parameters first using expectation-maximization (EM).

2) Train the HMM parameters using EM.

Is this training process correct or am I missing something?

Improve this question
$\endgroup$
6

4 Answers 4

Reset to default
8
$\begingroup$

In the reference at the bottom $^*$, I see the training involves the following:

  1. Initialize the HMM & GMM parameters (randomly or using prior assumptions).

    Then repeat the following until convergence criteria are satisfied:

  2. Do a forward pass and backwards pass to find probabilities associated with the training sequences and the parameters of the GMM-HMM.

  3. Recalculate the HMM & GMM parameters - the mean, covariances, and mixture coefficients of each mixture component at each state, and the transition probabilities between states - all calculated using the probabilities found in step 1.

$*$ University of Edinburgh GMM-HMM slides (Google: Hidden Markov Models and Gaussian Mixture Models, or try this link). This reference gives a lot of details and suggests doing these calculations in the log domain.

Improve this answer
$\endgroup$
3
$\begingroup$

This paper[1] is absolute classic and has the whole HMM machinery for gaussian mixture laid out for you. I think it's fair to say Rabiner made the first important step in speech recognition with GMM in 1980s.

[1] Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2), 257-286.

Improve this answer
$\endgroup$
1
  • 3
    $\begingroup$ I am aware of this paper, and I have read it. But what you have written is not an answer, you just point me to paper, and does not answer my question. Also my question was not about Rabiner, so this sentence is "I think it's fair to say Rabiner made the first important step in speech recognition with GMM in 1980s.") is irrelevant to my question. $\endgroup$
    – notArefill
    Commented Sep 12, 2016 at 16:23
1
$\begingroup$

pomegranate is another python library that provides GMM and HMM with even better documents than hmmlearn. Currently I prepare transfer from hmmlearn to it. http://pomegranate.readthedocs.io/en/latest/GeneralMixtureModel.html

Improve this answer
$\endgroup$
1
  • $\begingroup$ I'm not seeing support for continuous observations in that package. Does it support continuous observations? I've found some packages that do gaussian HMMs and some that do continuous observation HMMs, but I can't find one that does both at once. $\endgroup$
    – Danny
    Commented May 1, 2019 at 20:53
-1
$\begingroup$

Assuming your HMM uses Gaussian Mixture, for parameters estimation, you perform forward and backward pass and update the parameters. The difference is that you need to include normal pdf mixture as the probability of observation given a state. So, for transition probability estimation, you do it just like a discrete observation HMM, but to re-estimate the mean, variance(or covariance matrix for multivariate case), and mixture weights, you introduce a new formula for probability of being in state i at time t with m-th mixture component accounting for the observation at t, which is simply normalized alpha*beta * normalized c*N(o,u,var) alpha and beta are the forward and backward formulas in Baum-Welch and c = m-th mixture weight while being in state i, o = observation at t, u = mean or mean vector, var = variance or covariance matrix

Improve this answer
$\endgroup$
3
  • $\begingroup$ actually, the link given by an answerer Alex has pretty good description $\endgroup$
    – Kevvy Kim
    Commented Dec 29, 2016 at 6:10
  • $\begingroup$ Could you expand on how this adds to the previous answers? $\endgroup$
    – mdewey
    Commented Dec 29, 2016 at 9:37
  • $\begingroup$ If you mean train by re-estimating the parameters, then the page 18 of the link Alex has linked gives the equations. I could've given the equations here, but as I don't know how to put formulas, I would have a hard time explaining it in words. $\endgroup$
    – Kevvy Kim
    Commented Jan 2, 2017 at 13:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.