# Do hidden Markov models contain Markov chains?

Is it correct to say that the Hidden State Sequence in a Hidden Markov Model is a Markov Chain?

Thanks

-
 Hmm... The title asks if a Markov Chain exists as a component of a Hidden Markov Model, and the content of the question elaborates and asks whether that component is, in fact, the hidden state sequence. Thoughts? – Walrus the Cat May 30 '12 at 9:28

Yes, it's definitely a Markov chain because the observed variables define the transition probabilities from every latent variable configuration to every other. (That is, they yield an $n \times n$ matrix of probabilities where $n$ is the number of latent variable configurations.)

Usually you can say more about the Markov chain:

If you're talking about Boltzmann machines, for example, then the Markov chain you get from iterating the variables is time-homogenous since time doesn't matter. This would also be true in a regular Markov model if you were to hold the observed nodes fixed.

For Boltzmann machines, you also have ergodicity since all states are positive recurrent (for finite weights). You can often have the same property for a Markov model.

If you have an ergodic, time-homogenous Markov chain, then the network converges to a steady state distribution.

-

Yes, A hidden Markov model is a bivariate stochastic process $\{(X_t,Y_t)\}_{t\geq0}$, defined on a product space $(\mathsf{X} \times \mathsf{Y},\mathcal{X} \otimes \mathcal{Y})$, where $\{X_t\}_{t\geq0}$ is a Markov chain taking values in $\mathsf{X}$ and $\{Y_t\}_{t\geq0}$, defined on some state space $\mathsf{Y}$, is a sequence of observable random variables conditional on $\{X_t\}_{t\geq0}$.

-