Why are hidden Markov models considered 'mixture models'?

Question

I'm a bit confused by the conception of "mixture model" I'm studying hidden Markov model, which is frequently referred to as a "mixture model". But I don't know what the term "mixture" implies.

Suppose the HMM has state variable sequence $v_1,v_2,\dots,v_t$, $v_i\in\{1,\dots,k\}$. The emission model has pdf $f(y_i\mid\theta_{v_i})$. For each state $m$, the emission probability is a gaussian mixture $f(y\mid\theta_m)=\frac{1}{3}\mathcal{N}(\mu_m,\sigma^2)+\frac{1}{3}\mathcal{N}(\mu_m-10,\sigma^2)+\frac{1}{3}\mathcal{N}(\mu_m+20,\sigma^2)$

Why is it a mixture model? Is it because it has multiple states and each state has its own emission probability? Or is it because each state uses a Gaussian mixture?

According to Wikipedia, the second understanding is more probable. However, intuitively, I think the first understanding is more reasonable.

---------------added on July 13, 2015--------------------------

I know the definition of mixture model in Wikipedia. I (think I) know what is a mixture model. Now I need to know what is not a mixture model.

Is a multi-state model a mixture model in general?

Another evidence to support the first understanding. In the first three paragraphs of section 7, the paper claims

the observation $y_{t+1}$ is drawn from the mixture component indexed by $v_{t+1}$.

Here, $v_{t+1}\in\{1,\dots,k\}$ is a state variable, and given the realization of the variable, $f(y_{t+1}\mid\theta_{v_{t+1}})$ is an emission probability. Hence this multi-state model is called a "mixture model" and each probability (corresponding to a state), without being summed up, is called a "mixture component" in the paper.

Moreover, there might be a way to "mediate" the two understandings: if each state $m$ has an occurrence probability $\pi_m$, then, the marginal emission probability (state variable is integrated) is $\sum_{m=1}^k\pi_mf(y\mid\theta_m)$, this is obviously a mixture model.

Hence we have a general claim: a multi-state distribution can also be called a mixture model due to the above transformation.

Is this claim valid? Why?

Answer 1

A mixture distribution is a distribution made up of components which are themselves other distributions. The distribution is a mixture of its components.

The simplest case is the finite mixture, of which you give an example in your question.

$f(x) = p_1 f_1(x) + p_2 f_2(x) + ... + p_k f_k(x)$

$F(x) = p_1 F_1(x) + p_2 F_2(x) + ... + p_k F_k(x)$

where the mixing proportions $p_i$ sum to 1.

You can conceive of a mixture as first choosing a component with probability $p_i$ and then being generated from that component (indeed, that's a way to simulate from mixture distributions).

Here's the mixture you describe (with $\mu_m=50$ and $\sigma=5$):

---

Is a multi-state model a mixture model in general?

That depends on what you're looking at/how you frame it. If you look at the unconditional state (unconditional on earlier time), then it could be framed as a finite mixture of degenerate distributions (representing the states).

The paper itself states it quite explicitly (if framed slightly differently), right in the middle of the three paragraphs you pointed to:

This is essentially a dynamic variant of a finite mixture model, in which there is one mixture component corresponding to each value of the multinomial state.

Answer 2

A mixture model (MM) is a probabilistic model which represents subpopulations within the whole population, therefore it will approximate more closely data that is made up of two or more groups that have different individual distributions, than a model that assumes all data comes from one unique distribution.

In your HMM example, each state's emission probability is given by a gaussian mixture, which is the "mixture" part of the model.

This does not mean that all states have the same emission probabilities, since for each state the mean of each gaussian is different, hence the $\mu_m$ for each state m