I used hmmlearn library to initialize an HMM (Hidden Markov Model). sampled observations from the HMM, and used the sampled data to re-estimate the parameters of the HMM.

For re-estimating the parameters I randomly initialized the parameters and then used Baum-welch algorithm to learn the original parameters.
For 2 state HMMs, the re-estimated parameters are close to the original parameters. but for 3(or higher) state HMMs the re-estimated parameters are nowhere close to the original parameters. the log-likelihood of the re-estimated model is no where close to the log-likelihood of the original model. This indicates that I have gotten stuck in a local maxima.

All in all, I randomly initialized the parameters of the HMM. Then I tried to re-estimate parameters of the HMM using the Baum-welch algorithm. But, the Baum-welch algorithm always gets stuck in a local maxima.

Hence, I need a better way to initialize the HMM. After searching a lot, this is all that I could find.

  1. Initialising discrete HMMs: There is currently no clear way of initializing discrete hmms, that I know of, except a spectral algorithm suggested by D.Hsu .i have skimmed through his paper, but couldn't find any reliable package that implements it. Also, a look at this paper suggests that, implementing it wouldn't be a good idea(one of the reasons being that the spectral algorithm makes too many assumptions).

  2. Initialising HMMs with gaussian emissions: I guess, a GMM(Gaussian mixture model) could be used to learn/estimate the hidden states of the HMM, then the start probability vector and transition probability matrix could be learned in a supervised manner. Also, sklearn's implementation of GMMs is excellent.

  3. Initialising HMMs with Gaussian mixture emissions: I've got nothing.

so, does somebody/anybody, know a reliable method to initialize HMMs for the Baum-welch algorithm?

Also, finally, I have to apply hmms to a dataset(but not an NLP dataset). I don't know what type of datasets are suitable for hmms. I am thinking of time-series datasets. Is there anything specific I should look for in the dataset? do you know of a simple dataset, suitable for beginners?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.