# What is the difference between a Hidden Markov Model and a Mixture Markov Model?

From what I understand, Hidden Markov Models are those that relate observable and unobservable states, whilst Mixture Markov Models are techniques to cluster sequences according to which Markov model out of a set of them is able to approximate it better.

However, I am having trouble understanding boundaries between both terms. Can I define a Hidden Markov Chain without following the Mixture approach, and conversely, can I have a Mixture Model without considering hidden states?

Thanks for your help!

• please show me the source of your understanding of Mixture Markov models? Aug 20 '17 at 6:44

## 2 Answers

I am not familiar with what you call mixture Markov models. However, as I say further in this answer, some people call hidden Markov models, dynamical mixture model. It is possible that other people refer to Mixture Markov model then. I would be happy if you could indicate where you have read this term.

The previous answer to this question states things that are somewhat inaccurate about the relationship between mixture models and HMMs. What follows aims at clarifying this.

Mixtures models are simply a weighted sum of probability distributions, nothing more:

$P(X|\theta) = \sum_{i=1}^M w_i p_i(X|\theta_i)$, with $\sum_{i=1}^M w_i = 1$

$M$ is the number of components in the mixture. A random variable can follow a mixture model the same way it follows a probability distribution.

Hidden Markov models (HMM) are far more sophisticated models and mixture models can be a part of these models.

A HMM is based on a Markov Chain of states (said hidden states). It does not model a random variable but a time series (an ordered sequence of values, multidimensional or not). Each state of the Markov chain is associated with a probability distribution that can also be a mixture of distribution (here is the only point where the 2 concepts connect!)

At time $t=0$ a state is drawn from an initial probability mass function. The observation at time $t_0$ is assumed to follow (or to have been generated) by the probability distribution (or mixture depending on the case) associated to this drawn state. At time $t=1$, the system enters a new hidden state that is drawn from a transition matrix and the same procedure is repeated.

Hidden Markov models are sometimes called Dynamical Mixture Models (as in this technical report) because, when states are associated to mixtures of probability distributions, they can somehow be seen as a mixture model that dynamically changes over time. However, these changes are modeled via a transition matrix and are fully part of what a HMM is.

Opposite to what is said in the other answer, an HMM can be without mixture with any number of states (and even an infinite number of states). An HMM with a unique state is basically a mixture model, and if that mixture model has only 1 component, it is a simple probability distribution.

• The OP did say "Mixture Markov Model". While I'm not familiar with a technique of that specific name, I think it's clear that he/she wasn't talking about generic mixture models. Aug 28 '17 at 18:55
• Yes I got disturbed by the previous answer that inaccurately states things about HMMs which makes things very confusing. I was just aiming at clarifying this. I am going to add a point on this as I'm not familiar neither with any technique with this name (besides simple mixture models) Aug 28 '17 at 18:59
• What do you mean by 1 component of HMM? Can you explain how do u get output values from different probability distribution( from hidden state/states) without mixing it ? Aug 29 '17 at 7:13
• I agree with your good explanation of mixture model but please provide justification of your -1.( HMM without mixture model). We do need to have mixture to get output values. Aug 29 '17 at 7:17
• @ArpitSisodia "What do you mean by 1 component of HMM?" - I did not write this. I said that an HMM with a number of hidden states equal to 1 is equivalent to a mixture model (it is basically equivalent to the emission distribution associated with this unique state). A mixture with a number of components equal to one (degenerate case) is a simple probability distribution. You understanding of HMM is not accurate from what you write. You are confusing the transition matrix that allows to transit between states and the emission probabilities that can be mixture models or simple distributions. Aug 29 '17 at 14:04

I dont think there is any difference between Hidden Markov Model( HMM) and Mixture Markov Model. There are HMMs and Gaussian Mixture models( GMMs).Please show me where have you read about any difference?

HMM is a kind of mixture model when you have to calculate output value from the hidden states. For example you have 2 hidden states h1 and h2 with participation probabilities of .3,.7 respectively. Lets say, The output value distribution from h1 is p1 and h2 is p2. Thus using concept of mixture model, you get output value as-

.3 X p1 + .7 X p2 ( which is nothing but the combination of 2 different distributions; mixture model).

Mixture Models have nothing to do with hidden states. Hidden markov model uses, forward algo, Viterbi algo and baum welch algo in its implementation, and gives its parameters; transition +emission probabilities+ viterbi sequence( states participation probabilities) . You have to have a fixture model if you want to get output value from these parameters.Like in our example it was- .3* emission distribution1 + .7* emission distribution2)

• "HMM can be without mixture if hidden state is 1" - No, HMMs can be without mixtures with any number of states Aug 28 '17 at 19:03
• @Eskapp , how will you get output values. Without mixing different output distribution from hidden states. Aug 29 '17 at 7:07
• Check my comment in my answer. I'll remove the -1 vote when you correct your answer. It was meant to make you react so we are able to have this discussion :) As I can see from what you write, you have some misunderstanding/confusion with the HMM concept. It's not an easy model to understand and I would like to prevent spreading inaccuracies about these models. Aug 29 '17 at 14:11
• I know it's not easy to understand,but I have worked on many HMM models in Pharma and do have HMM implementation knowledge end to end :) . Perhaps you are not clear about different steps in HMM and just have overview of it. Please go through these links/papers- machinelearningstories.blogspot.in/2017/02/… ,machinelearningstories.blogspot.in/2017/03/… Aug 30 '17 at 5:49
• @Eskapp- please let me know if u have any doubt in understanding this paper. cs.umb.edu/~rvetro/vetroBioComp/HMM/… , I had done a lot of research in HMM. Aug 30 '17 at 5:51