# Hidden-Markov Model for Markov-Chain with Sequential Bernoulli State Sampling

Consider a finite discrete-time Markov chain whose state is sampled at the times determined by the outcome of a Bernoulli process. That is, in each time period I flip a biased coin. If it comes up as "heads," I get to see the state of the Markov chain. If it comes up as "tails," I see nothing. My questions are:

1. Suppose the Bernoulli process is independent of the chain's state. Can one describe this observation model using a hidden Markov model specification?

2. If the answer to the above is "yes", is it possible to code this specification in depmixS4 or other HHM package to fit the parameters of the chain and the Bernoulli process?

3. How are the answers to the above affected if the probability of success in the Bernoulli process is influenced by the state of the chain?

• Was this a homework question? – Placidia Aug 23 '19 at 17:42
• No, it is not a homework question. It is a reduction of a more complex model specification. – user3605620 Aug 23 '19 at 17:49
• @placidia I haven read past your first paragraph below, but I think this alone somewhat adds to the confusion. You’re avoiding the definition about the type of model that’s being asked about, and you’re using terminology that isn’t used very often in the HMM world. In particular, MAR and MCAR are used in situation where the distribution of whether the data is missing is conditional on the data that would’ve been generated. HMMs in the other hand tend to have deterministic patterns of missingness, and the whether isn’t even discussed. (+1) on the question, though. – Taylor Aug 23 '19 at 18:36

For the transition probabilities of the Markov process, you could simply write down the likelihood of everything you see and throw the result at your favourite optimizer. This should not be difficult. For example, if $$p_{ij}$$ is the probability of transitioning from state $$i$$ to state $$j$$ in one time step, then you would include such a term in the likelihood every time you see the process transition from $$i$$ to $$j$$ in one time step. If you observe a transition from $$i$$ to $$j$$ in two time steps, with the middle step missing, include a term like $$\sum_k p_{ik}p_{kj}$$. And so on. It might be less confusing to work with a matrix representation.