# the combination of two independent continuous time Markov chains

Assume one variable $x$ has two states 0 and 1, $x$ changes between 0 and 1 following a continuous time Markov chain. The transition probability is represented as matrix $P$ and the time sojourning on a state follows exponential distribution with arrival ratio $\lambda$.

There is the other variable y which is independent with $x$. $y$ also changes between 0 and 1, and have the same $P$ and $\lambda$ with $x$.

Now the question is: will the new variable $z=x*y$ follow continuous time Markov chain? if yes, what will be the transition probability and $\lambda$. ($z$ only has two states: 0 and 1.)

I guess $z$ still follows continuous time Markov chain, but I have no idea how to prove? (maybe there has been a property stating this, which I don't know?) Could anyone give some ideas? Many thanks!

• What does $*$ represent? – Neil G Mar 25 '13 at 22:22
• "*" simplify means multiplication. – ulyssis2 Mar 26 '13 at 8:04

The product $z_t$ is not a Markov Chain (MC).
Assume we know that $z_{t_0} = 0$ at some time $t_0$, meaning that $x_{t_0}=0$ or $y_{t_0}=0$. Then the past values $\{z_{u};\,u <t_0\}$ of $z_t$ still contain information about future values $\{z_v;\, v > t_0\}$ in contradiction with the Markov property. Indeed, let $s_t := [x_t, \,y_t]$; so that $s_t$ is a MC taking the $4$ values written here as $00$, $01$, $10$ and $11$. At time $t_0$, only the first $3$ states are possible since the product is $0$. Let $t_0-W$ be the random time of the latest state change for $z_t$, which was $z: \,1 \rightarrow 0$. If $W$ is small, then most probably $s_{t_0}$ is $01$ or $10$, but not $00$: only one change of state occured during the interval $(t_0-W, \,t_0)$. This in turn tells us that the next transition $0 \rightarrow 1$ will occur more quickly than if a large value $W$ had been obtained.
Here is a more formal derivation. For a fixed $t$ and a small $h > 0$ we have $$\mathrm{Pr}\{z_{t-h} = 1\,\vert \,s_{t} = 00\} = o(h), \qquad \mathrm{Pr}\{z_{t-h} = 1\,\,\vert \, s_{t} = 01\} = \lambda h + o(h)$$ with $\lambda >0$. Indeed, the first probability involves two transitions of $s_t$. Using Bayes formula $$\mathrm{Pr}\{s_{t} = 00 \,\vert\, z_{t-h} = 1, \, z_{t}=0\} = \frac{\mathrm{Pr}\{ z_{t-h} = 1 \,\vert \,s_{t} = 00\} \, \mathrm{Pr}\{ s_{t} = 00\, \vert \, z_{t} = 0\}}{ \mathrm{Pr}\{ z_{t-h} = 1 \,\vert \, z_{t} = 0 \} }.$$ The numerator of the fraction is $o(h)$, while its denominator is easily found to be $\nu h + o(h)$ for some $\nu >0$, so the probability is $o(h)$. By contrast $$\mathrm{Pr}\{s_{t} = 01 \,\vert\, z_{t-h} = 1, \, z_{t}=0\} = \rho h + o(h)$$ for some $\rho > 0$. Thus conditional on $\{z_{t-h} = 1, \, z_{t}=0\}$ the event $\{s_{t} = 01\}$ is much more probable than $\{s_{t} = 00\}$ for small $h$, as claimed.
Here $z_t$ follows a Hidden Markov Model (HMM) with hidden state $s_t$ and simply results from grouping $3$ possible states of $s_t$ as one $z_t=0$. More generally, grouping states of a MC $s_t$ will result in a HMM process but not a MC.
• Happy if this helps... Yes, at $t_0$ the time spent in state $z=0$ contains information about how quickly we can leave it. One can draw a graph with the four states of $s_t$ as nodes and transitions as arcs. Then $z_t=0$ means that $s_t$ is in the group of states $\{00,\,01, 10\}$. Within this group, $s_t$ can be one step away from the exit $z =0 \rightarrow z=1$, or two steps away. – Yves Mar 26 '13 at 18:40
• Since the two MC $x_t$ and $y_t$ are time homogeneous so is the MC $s_t$. The time spent by $s_t$ in the state $11$, which is the time spent by $z_t$ in state $1$, can be computed using continuous MC theory. This imply: first writing the $4 \times 4$ generator matrix for $s_t$ using the two rates $\lambda$ and $\mu$ for transitions $0 \rightarrow 1$ and $1 \rightarrow 0$, then find the stationnary distribution. From a more more statistical point a view, if only $z_t$ is observed, inference on $x_t$ and $y_t$ can be drawn using HMM techniques (Baum-Welch, Viterbi, ...). – Yves Mar 29 '13 at 8:56