2
$\begingroup$

I used hmmlearn library to initialize an HMM (Hidden Markov Model). sampled observations from the HMM, and used the sampled data to re-estimate the parameters of the HMM.

For re-estimating the parameters I randomly initialized the parameters and then used Baum-welch algorithm to learn the original parameters.
For 2 state HMMs, the re-estimated parameters are close to the original parameters. but for 3(or higher) state HMMs the re-estimated parameters are nowhere close to the original parameters. the log-likelihood of the re-estimated model is no where close to the log-likelihood of the original model. This indicates that I have gotten stuck in a local maxima.

All in all, I randomly initialized the parameters of the HMM. Then I tried to re-estimate parameters of the HMM using the Baum-welch algorithm. But, the Baum-welch algorithm always gets stuck in a local maxima.

Hence, I need a better way to initialize the HMM. After searching a lot, this is all that I could find.

  1. Initialising discrete HMMs: There is currently no clear way of initializing discrete hmms, that I know of, except a spectral algorithm suggested by D.Hsu .i have skimmed through his paper, but couldn't find any reliable package that implements it. Also, a look at this paper suggests that, implementing it wouldn't be a good idea(one of the reasons being that the spectral algorithm makes too many assumptions).

  2. Initialising HMMs with gaussian emissions: I guess, a GMM(Gaussian mixture model) could be used to learn/estimate the hidden states of the HMM, then the start probability vector and transition probability matrix could be learned in a supervised manner. Also, sklearn's implementation of GMMs is excellent.

  3. Initialising HMMs with Gaussian mixture emissions: I've got nothing.

so, does somebody/anybody, know a reliable method to initialize HMMs for the Baum-welch algorithm?

Also, finally, I have to apply hmms to a dataset(but not an NLP dataset). I don't know what type of datasets are suitable for hmms. I am thinking of time-series datasets. Is there anything specific I should look for in the dataset? do you know of a simple dataset, suitable for beginners?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.