Hidden Markov Models with two emissions per state

Question

I have a problem formulated in terms of hidden markov models. A simplified version is discussed here. I have a system that transitions between two discrete hidden states (states 1, 2). The transition probabilities between these states are known, and are shown in the figure below:

Moreover, I have two instruments (I1 and I2) independently observing the system, trying to classify the system as being in state 1 or 2 at each time interval. The observations by the instruments are made at discrete time intervals (say, every hour). Instrument I1 is more dependable than I2. However, data from I1 is not available at all time intervals.

The dependability of the instruments is also estimated independently, with values such as

P(S1 | I1=1) = 0.9
P(S2 | I1=1) = 0.1
P(S1 | I1=2) = 0.8
P(S2 | I1=2) = 0.2
..etc, for instrument 2 too...

where P(S1 | I1=1) is the probability that the actual state was '1' given that data from instrument 1 estimated it to be '1'.

Now, I have observations (and estimated states) from both the instruments for a given time period:

Time_period I1   I2 
1            1     1 
2            NA    2 
3            2     2 
4            1     2 
5            1     1 
6            NA    2 
7            NA    2 
8            1     2 
9            1     1

where I1 is the state inferred from the observation of instrument 1 and I2 is the state inferred from the observation of instrument 2, and NA means that the observation was not available. Also, I have initial values of what the states were. Given these and the transition probabilities, I would like to get the posterior probability of the two states. That is, something like this (below):

Time_period  I1   I2   pp_1     pp_2
1            1     1   0.92     0.08
2            NA    2   0.85     0.15
3            2     2   0.20     0.80
4            1     2   0.70     0.30    
5            1     1   0.90     0.10
6            NA    2   0.95     0.05
7            NA    2   0.93     0.07
8            1     2   0.90     0.10
9            1     1   0.95     0.05

where pp_1 is the posterior probability that the actual state was state 1 and pp_2 is the probability that the actual state was state 2.

I know that this post talked about a similar problem. But I am not sure if I can use the msm package, as I have a discrete time situation. I would like to code this in R; which R package is best for this? Can someone please guide me, maybe with a few lines of relevant code?

Answer 1

I think I have an answer to this at this point. The first step is to assign probabilistic values for each state at each time step, given just the observations. For example, for the first time step, both I1 and I2 had estimated the state as '1'. Now, given what we know about the accuracy of these instruments, what is the probability that the state was actually '1'? This is the problem of combining multiple pieces of evidence. For a good example, see here: (see formula (5) for the final formula). For the time periods when an instrument output is missing, we use the probability estimated from just one instrument.

Thus, we calculate this kind of table by combining the evidence from both the sensors. So, we may have something like:

Time_period I1   I2      (prior) probability of state 1    (prior) probability of state 2
1            1     1         0.88                               0.12
2            NA    2         0.20                               0.80
3            2     2         0.07                               0.93
4            1     2        ...etc...
5            1     1 
6            NA    2 
7            NA    2 
8            1     2 
9            1     1

NOTE! The probability numbers shown in the above table are just illustrations, and NOT actual calculations!

Again, if BOTH instrument outputs are missing for some time step, we can use a 'best guess' estimate (say, probabilities of 0.5 for both state1 & state2).

The second part (and again, I am not sure of this part!) is that you use a method known as 'HMM smoothing', as described in 'Machine Learning: A Probablistic Perspective' by Kevin Murphy (2012).

In that textbook, chapter 17 is 'Markov and hidden markov models'. In 17.4.1, the 'smoothing' problem is explained. The problem I have described maps well into such a smoothing problem. In 17.4.3, the forwards-backwards algorithm is presented as a way to solve the smoothing problem.

Hope this helps someone!

Answer 2

To use msm, you would have to assume an underlying continuous-time model, and find transition intensities in continuous time which are equivalent to the transition probabilities of 0.7 and 0.9 in discrete time. This may or may not be possible. The intensity matrix Q is the matrix logarithm of the probability matrix P - I don't have experience with matrix logs, but I think there's a risk they would have multiple/nonexistent solutions. So I'd advise you find software which fits the discrete time model that you want. Afraid I haven't used any of the R packages for discrete time HMMs so can't advise any further.