# What's a multivariate Hidden Markov Model?

I suppose "multivariate" refers to a function w/ multiple inputs. What function? One that, given an output sequence of a HMM, returns the probability distribution of state switch sequences of the HMM for the output? Something else?

Suppose we've 3 outputs: ax, ab, Ax. Would a multivariate (bivariate) HMM rather refer to f(ax, ab) or d(character, capitalization)? The first would define a probability (distribution) of state switch sequence(s), the second a set of output symbols. Of course, both would be equivalent to monovariant HMMs (e.g. the first to two evaluations of a unary function computing the same state switch probability (distribution) for ax and ab, the second to a function over an extended character set)

We usually call multivariate HMM an HMM that model multidimensional observations.

If you have time series in the form:

X = [1 2 3 4 5 1 2] (each value corresponding to a time step), you will model them using a univariate HMM (as only one variable varies)

If the time series you are modeling have the form: X = [1 2 3 4 5 1 2; 5 4 3 2 1 5 4] (i.e., a matrix with 2 rows, with each column corresponding to a time step), you will model them using a multivariate HMM (as you are observing multiple variables - 2 in this case).

The dimensions of your observations defines the dimension of the emission probability distributions used in the HMM.

• OK.. then my example w/ d(character,capitalization) would describe a multivariate HMM? In the last sentence, did you mean "...defines the dimensions of the emission..."?
– jaam
Nov 11 '16 at 18:02
• @jaam - An HMM has hidden states. Each hidden state is associated with a probability mass function (discrete case) or a probability distribution function (continuous case). This is what is also called "emission distribution" (the distribution are assumed to generate/emit the observations). The transition between the hidden states is modeled by a transition matrix which has nothing related to the distributions functions assigned to the hidden states. Nov 15 '16 at 16:52
• @jaam - I suggest you to read this: ai.stanford.edu/~pabbeel/depth_qual/Rabiner_Juang_hmms.pdf It should answer your concerns with some very concrete examples. Best introduction to HMMs that I know ;) Nov 15 '16 at 16:53
• Consider a matrix [a, ..., z; lowercase, uppercase]. As long as character and capitalization are types and d a function variable, the matrix is clearly computable by some d(character,capitalization). Thus, some d(character,capitalization) would describe a multivariate HMM. If not, why?
– jaam
Nov 19 '16 at 18:03

In a univariate hidden Markov model, one emission corresponds to one random variable, whereas in a mulivariate hidden Markov model, one emission corresponds to several random variables.

• "Correspond" is vague. Do you mean that variables are state switch sequences, and a multivariate HMM maps multiple state switch sequences to an emission? In a multivariate HMM, does every emission correspond to multiple state switch sequences? Or do you mean that the multiple variables the emission can correspond to are e.g. x, uppercase?
– jaam
Nov 10 '16 at 21:06