3
$\begingroup$

Assume one variable $x$ has two states 0 and 1, $x$ changes between 0 and 1 following a continuous time Markov chain. The transition probability is represented as matrix $P$ and the time sojourning on a state follows exponential distribution with arrival ratio $\lambda$.

There is the other variable y which is independent with $x$. $y$ also changes between 0 and 1, and have the same $P$ and $\lambda$ with $x$.

Now the question is: will the new variable $z=x*y$ follow continuous time Markov chain? if yes, what will be the transition probability and $\lambda$. ($z$ only has two states: 0 and 1.)

I guess $z$ still follows continuous time Markov chain, but I have no idea how to prove? (maybe there has been a property stating this, which I don't know?) Could anyone give some ideas? Many thanks!

$\endgroup$
  • $\begingroup$ What does $*$ represent? $\endgroup$ – Neil G Mar 25 '13 at 22:22
  • $\begingroup$ "*" simplify means multiplication. $\endgroup$ – ulyssis2 Mar 26 '13 at 8:04
4
$\begingroup$

The product $z_t$ is not a Markov Chain (MC).

Assume we know that $z_{t_0} = 0$ at some time $t_0$, meaning that $x_{t_0}=0$ or $y_{t_0}=0$. Then the past values $\{z_{u};\,u <t_0\}$ of $z_t$ still contain information about future values $\{z_v;\, v > t_0\}$ in contradiction with the Markov property. Indeed, let $s_t := [x_t, \,y_t]$; so that $s_t$ is a MC taking the $4$ values written here as $00$, $01$, $10$ and $11$. At time $t_0$, only the first $3$ states are possible since the product is $0$. Let $t_0-W$ be the random time of the latest state change for $z_t$, which was $z: \,1 \rightarrow 0$. If $W$ is small, then most probably $s_{t_0}$ is $01$ or $10$, but not $00$: only one change of state occured during the interval $(t_0-W, \,t_0)$. This in turn tells us that the next transition $0 \rightarrow 1$ will occur more quickly than if a large value $W$ had been obtained.

Here is a more formal derivation. For a fixed $t$ and a small $h > 0$ we have $$ \mathrm{Pr}\{z_{t-h} = 1\,\vert \,s_{t} = 00\} = o(h), \qquad \mathrm{Pr}\{z_{t-h} = 1\,\,\vert \, s_{t} = 01\} = \lambda h + o(h) $$ with $\lambda >0$. Indeed, the first probability involves two transitions of $s_t$. Using Bayes formula $$ \mathrm{Pr}\{s_{t} = 00 \,\vert\, z_{t-h} = 1, \, z_{t}=0\} = \frac{\mathrm{Pr}\{ z_{t-h} = 1 \,\vert \,s_{t} = 00\} \, \mathrm{Pr}\{ s_{t} = 00\, \vert \, z_{t} = 0\}}{ \mathrm{Pr}\{ z_{t-h} = 1 \,\vert \, z_{t} = 0 \} }. $$ The numerator of the fraction is $o(h)$, while its denominator is easily found to be $\nu h + o(h)$ for some $\nu >0$, so the probability is $o(h)$. By contrast $$ \mathrm{Pr}\{s_{t} = 01 \,\vert\, z_{t-h} = 1, \, z_{t}=0\} = \rho h + o(h) $$ for some $\rho > 0$. Thus conditional on $\{z_{t-h} = 1, \, z_{t}=0\}$ the event $\{s_{t} = 01\}$ is much more probable than $\{s_{t} = 00\}$ for small $h$, as claimed.

Here $z_t$ follows a Hidden Markov Model (HMM) with hidden state $s_t$ and simply results from grouping $3$ possible states of $s_t$ as one $z_t=0$. More generally, grouping states of a MC $s_t$ will result in a HMM process but not a MC.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Thanks a bunch! This is inspiriting! In a short saying against that z follows CTM, the next new state of z after t_0 is not only decided by the its state on t_0, but the transition process which leads to its state on t_0. $\endgroup$ – ulyssis2 Mar 26 '13 at 17:04
  • 1
    $\begingroup$ Happy if this helps... Yes, at $t_0$ the time spent in state $z=0$ contains information about how quickly we can leave it. One can draw a graph with the four states of $s_t$ as nodes and transitions as arcs. Then $z_t=0$ means that $s_t$ is in the group of states $\{00,\,01, 10\}$. Within this group, $s_t$ can be one step away from the exit $z =0 \rightarrow z=1$, or two steps away. $\endgroup$ – Yves Mar 26 '13 at 18:40
  • $\begingroup$ although z does't follow CTM, can it be modeled by some other models? I am now curious about the duty circle (the proportion of state 1 in a long duration) of z. $\endgroup$ – ulyssis2 Mar 26 '13 at 19:22
  • $\begingroup$ Since the two MC $x_t$ and $y_t$ are time homogeneous so is the MC $s_t$. The time spent by $s_t$ in the state $11$, which is the time spent by $z_t$ in state $1$, can be computed using continuous MC theory. This imply: first writing the $4 \times 4$ generator matrix for $s_t$ using the two rates $\lambda$ and $\mu$ for transitions $0 \rightarrow 1$ and $1 \rightarrow 0$, then find the stationnary distribution. From a more more statistical point a view, if only $z_t$ is observed, inference on $x_t$ and $y_t$ can be drawn using HMM techniques (Baum-Welch, Viterbi, ...). $\endgroup$ – Yves Mar 29 '13 at 8:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.