HMM for multichannel - multivariate data

Question

I have a dataset which contains multiple time series of multivariate data both continuous and categorical

I would like to use a Hidden Markov Model (HMM) to find the number of hidden states for all of my data. So I want to use all the time series that I have in order to build my HMM.

I have seen libraries in r like seqHMM that can handle multiple time series but not for heterogenous data (i.e. multivariate data that contains continuous and categorical data).

On the other hand libraries like depmixS4 can handle heterogenous multivariate data but for only one time series a time.

So since I cannot find a library that covers all my conditions I am wondering what procedure should I follow in order to build an HMM using my data.

For example can I do some preprocessing in my data and then use one of those libraries. Is there some technique/library I could use?

Thanks a lot for your time

Answer 1

One option:

I worked on this exact same problem couple of years ago. The mixed data I was working with were multivariate with some of the dimensions being categorical and some other continuous.

The trick has been to modify the equations of the Baum-Welch approach so it can handle both types of data. For instance, the responsibilities (outputs of the forward-backward algorithm) are approximated from "local responsibilities" that are computed with respect to each type of input (then considering data of smaller dimension then = number of dimensions of a sample being of one type). Then these approximated responsibilities are used for updating the HMM parameters. The distributions parameters though, have to be updated with respect to data of each type. Then a "global" HMM likelihood can be computed for convergence checking.

This was part of my PhD work, and the entire approach is reported in this paper:

Hybrid hidden Markov model for mixed continuous/continuous and discrete/continuous data modeling. E. Epaillard, N. Bouguila, MMSP'15, pp. 1-6

You will find no library for this as no method existed at the time I did this work (at least to the best of my knowledge). I was planning to release all my codes in the next few weeks, as I am completing my degree. My implementation is in Matlab and my code quite not documented at this time. However, if this could be of some help for you, I could release the code earlier. Just ask me in the comments.

Another option:

Also, if your data is of mixed types due to the fact it comes from different sources of information. Then I would have a look at multi-stream HMMs. I am not sure whether these models work for discrete/continuous mixed data though, but in the end, I found my approach to be somehow related to the theory behind multi-stream HMMs.

Multi-stream HMMs seems to also be a type of model that is so far mostly used in research and not implemented in main libraries (let's remember that it is already hard to find HMM other than Gaussian-based in libraries...). I find the theory behind these models quite difficult to understand (a lot of optimization going on). Maybe you have a chance to find some implementation by going through the list of the authors of the main papers on the topic, go through their personal websites and see whether they shared their codes or not.