# Hidden Markov Models with multiple emissions per state

I want to use Hidden Markov Models for an unsupervised sequence tagging problem. Due to the peculiarities of my application domain (recognition of dialogue acts in conversations), I would like to use multiple emissions for each state (that is, multiple features). Graphically, the model would therefore look like this:

Both the hidden states and the observation variables are discrete. The emissions probabilities $P(O_{ij} \ | \ S_i)$ are assumed to be independent and modelled via standard categorical distributions.

My question is the following: are there any publicly available toolkits or algorithm that would allow me to learn the parameters of such type of multiple-emissions HMM through a variant of Baum-Welch? From what I could gather, it seems that the only type of multiple emissions supported by classical HMM toolkits are multivariate Gaussians, but I could not find anything about independent categorical distributions of the type above.

Of course, I am aware I could "bypass" the problem by considering each observation to be a vector of values (with each dimension in this vector corresponding to a particular feature) and estimating emission probabilities on this vector space through classical Baum-Welch, but that would introduce a lot of unnecessary data sparsity.

Does anybody have a suggestion to solve this issue? I'm sure I'm not the first person that tried to apply HMMs for unsupervised learning with multiple features! (or maybe I should use another type of model? I considered using CRFs as well, but they seem tricker to apply to unsupervised learning problems).

• this is a common sitation in animal movement models (where we typically have turning angle and step length for each observation); github.com/benaug/move.HMM handles this, as I think does the depmixS4 R package (more general). – Ben Bolker Aug 27 '14 at 22:30
• search google for "hmm multivariate observations" and you will find many educational sites and research publications that cover this topic. e.g. datalab.uci.edu/papers/kirshner_thesis.pdf – kgierach Feb 10 '16 at 23:08