If I draw the computational graph of an HMM and an RNN from an architectural point of view they look very similar. The main difference is that an RNN gets some input $x$ and the HMM only operates on the latent sequence. From my understanding - and reading that from the drawn graph - both the hidden states $h_i$ in the RNN and the latent variable $Z_i$ are conditionally dependent on the previous state and produce some output. What I don't understand is, that an HMM is said to have the markov property (i.e., is memoryless) whereas the RNN does not. Why, where can I observe the difference? How can I read that property from a computational graph (if at all)?


1 Answer 1


The Markov property applies to sequences of random variables. Let $(X_n)_{n\in\mathbb{N}^*}$ be a sequence of random variables, it is said to follow the Markov property iif: $$ p(x_n|x_1,\dots,x_{n-1})=p(x_n|x_{n-1}). $$

Thus, if you can show, from the computational graph or any other equations defining your model, that the distribution of $X_n$ given $X_1,\dots,X_{n-1}$ only depends on $X_{n-1}$ you have proven the Markov property for the sequence of r.v. in question. The Markov property for the hidden random variables in a HMM is indeed one of the core definitions of HMMs.

RNNs -in the simplest models- have indeed architectures which seem to exhibit a Markovian property. I think however that this term is not really used because hidden variables in RNN are not random variables. Hence there might be a problem of definition; even though it is true that the dependency on the past states is reduced to the depency on the last state (at least in the most simple RNNs).

As a final remark, if you generalize RNNs with stochastic hidden random variables (you can see the deterministic case as a particular case) then you will be able to treat it as some kind of HMM and properly show the Markovian property (or not). See for example : here or here.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.