Hidden markov models: output observations defined by a (non-hidden) markov model?

Question

Let me explain what my goal is: I would like to define a hidden markov model with two hidden states and say, five possible observations. As I understand (I'm quite new to HMMs), in each state HMM will output one of the observations, based on the given output probabilites. I would like to extend this behaviour in such a way that the output is not dependant only on that probabilities, but also on the last observation that was output by the HMM.

You could think of it as if each hidden state had another (non-hidden) markov model embedded in itself, and use it to define the output. Both hidden states would have exactly the same markov models embedded, but with different transition probabilities.

I hope it is clear what I'm asking. Any hint would be greatly apreciated.

Answer 1

What you can do, to fit it into the classical HMM framework, is define a new hidden state space equal to the product of your current two state space and the output space. Given five possible outputs, this will result in a state space with 10 states. Let us define $S_t$ as the state at time $t$ and $O_t$ as the output at time $t$, which I'll assume can take on values $1, 2, \dots, 5$ for simplicity. $S_t = 1$ might correspond to your original state 1 and $O_{t-1} = 1$, $S_t=2$ to original state 1 and $O_{t-1}=2$, and so forth.

Transition probabilities - The transitions between states $S_t$ and $S_{t+1}$ are very limited once you've seen the output at time $t$, as there are only two states in the state space which it is possible to transition into. You can retain the simplicity of your current transition matrix by making the assumption that the transition probability from $S_t$ to $S_{t+1}$ depends solely on the "part" of $S_t$ that is due to your original definition of state. If you make this assumption, which seems to be what you want to do, you will only have two transition probabilities to estimate, and consequently will not suffer from an explosion of the number of transition probabilities in your model.

Answer 2

This is easily handled with a Markov model, since it is OK to have the outcome a time T be dependent on the state at T-1. What would not be OK is for the outcome to be dependent on previous state.

Reducing this to its simplest form, consider a Markov chain.

A Markov chain, named after Andrey Markov, is a mathematical system that undergoes transitions from one state to another, between a finite or countable number of possible states. It is a random process characterized as memoryless: the next state depends only on the current state and not on the sequence of events that preceded it. This specific kind of "memorylessness" is called the Markov property.

Here is a sample: