# Hidden-Markov Model for Markov-Chain with Sequential Bernoulli State Sampling

Consider a finite discrete-time Markov chain whose state is sampled at the times determined by the outcome of a Bernoulli process. That is, in each time period I flip a biased coin. If it comes up as "heads," I get to see the state of the Markov chain. If it comes up as "tails," I see nothing. My questions are:

1. Suppose the Bernoulli process is independent of the chain's state. Can one describe this observation model using a hidden Markov model specification?

2. If the answer to the above is "yes", is it possible to code this specification in depmixS4 or other HHM package to fit the parameters of the chain and the Bernoulli process?

3. How are the answers to the above affected if the probability of success in the Bernoulli process is influenced by the state of the chain?

• Was this a homework question? – Placidia Aug 23 '19 at 17:42
• No, it is not a homework question. It is a reduction of a more complex model specification. – user3605620 Aug 23 '19 at 17:49
• @placidia I haven read past your first paragraph below, but I think this alone somewhat adds to the confusion. You’re avoiding the definition about the type of model that’s being asked about, and you’re using terminology that isn’t used very often in the HMM world. In particular, MAR and MCAR are used in situation where the distribution of whether the data is missing is conditional on the data that would’ve been generated. HMMs in the other hand tend to have deterministic patterns of missingness, and the whether isn’t even discussed. (+1) on the question, though. – Taylor Aug 23 '19 at 18:36

## 1 Answer

I suppose it depends on how you define a HMM, but to me, this does not qualify. It's a Markov process with MCAR missing values. The parameters should be easy enough to estimate. Best estimate of the probability of the Bernouilli process is the proportion of observations to total time.

For the transition probabilities of the Markov process, you could simply write down the likelihood of everything you see and throw the result at your favourite optimizer. This should not be difficult. For example, if $$p_{ij}$$ is the probability of transitioning from state $$i$$ to state $$j$$ in one time step, then you would include such a term in the likelihood every time you see the process transition from $$i$$ to $$j$$ in one time step. If you observe a transition from $$i$$ to $$j$$ in two time steps, with the middle step missing, include a term like $$\sum_k p_{ik}p_{kj}$$. And so on. It might be less confusing to work with a matrix representation.

I am not familiar with depmixS4.

Absolutely nothing changes if the probability of an observation depends on the state. In this case, the data are missing at random (MAR) instead of missing completely at random (MCAR). This does not bias the likelihood function. If the dependence upon state were so severe that some states were hardly ever observed, there would be a higher variance on the estimates of transitional probabilities involving those states, but a weak estimate is not the same thing as a biased one.

Note that the scenario you raise occurs frequently in medical studies about disease progression. Patients may be scheduled more frequent visits to the physician as their condition worsens. However, being seen by a physician does not in itself change the progression of the disease, which is caused by the disease itself, the patient's physique and prescribed treatment.

• Thanks for the answer. I 100% agree that the setting is quite common. I also agree that LL function is easy to derive. However, ideally, there is a package that can fit this model. I do have additional covariates in the data making coding the parameter estimation problem from scratch a bit messy. So, any suggestions on the packages? – user3605620 Aug 23 '19 at 18:10
• You could check out R package msm. It allows for covariates as well as situations where the events occur at variable times. – Placidia Aug 25 '19 at 2:05