0
$\begingroup$

Consider a generator matrix $Q\in\mathbb{R}^{h\times h}$ for a discrete state space $\{1,...,h\}$. I want to determine the probability of a single transition of a stochastic process $X(t)$ with $X(0)=i$ and $X(\Delta t)=j$ with both $i,j\in\{1,..,h\}$.

The transition probability matrix during timestep $\Delta t$ is found by taking the matrix exponential of the generator matrix, thus $P_{\Delta t,i\rightarrow j}=\exp(Q\cdot \Delta t)_{i,j}$. The transition probability matrix takes into account every possible path from $X(0)=i$ to $X(\Delta t)=j$. $P_{\Delta t,i\rightarrow j}$ includes the probability of $X(t)$ undergoing multiple transitions between $0$ and $\Delta t$ with $X(0)=i$ and $X(\Delta t)=j$, for example $X(0)=i$, $X(\Delta t/2)=h$ and $X(\Delta t)=j$ with $i\neq h\neq j$. The probability of $P_{\Delta t,i\rightarrow j}$ is different than the probability of $X(t)$ undergoing one single transition from $i$ to $j$ for $t\in[0,\Delta t)$.

In the derivation of the generator matrix and corresponding transition probability matrix we have several useful definitions. The stochastic process $X(t)$ with $X(0)=i$ remains in state $i$ for a random amount of time, which is exponentially distributed, thus $\sim q_{i}\exp(-q_{i}\cdot\Delta t)$ with $q_{i}=-q_{ii}$. Given a transition from state $i$ at $t\in[0,\Delta t)$ the conditional probability is defined as $\mathbb{P}(X(t)=j|X(t^{-})=i,X(t)\neq i)=\frac{q_{ij}}{q_{i}}$ with $q_{ij}=\lim_{\Delta t\rightarrow0}\frac{\mathbb{P}(X(\Delta t)=j|X(0)=i)}{\Delta t}$.

My attempt was to simulate for every timestep $\Delta t$ the probability of one single transition of $X(t)$ from state $i$ to $j$ by $q_{i}\exp(-q_{i}\cdot t)\frac{q_{ij}}{q_{i}}\exp(-q_{j}(\Delta t-t))=q_{ij}\exp(-q_{i}\cdot t)\exp(-q_{j}(\Delta t-t))$ for $t\in[0,\Delta t)$. Next, probabilities for all possible values of $t\in[0,\Delta t)$ must be summed. By integrating over $t$, I find $$ \begin{split} \mathbb{P}(X(t^{+}=j),X(t^{-}=i\neq j),t\in[0,\Delta t)|X(0)=i)&=\int^{\Delta t}_{0}q_{ij}e^{-q_{i}\cdot t}e^{-q_{j}(\Delta t-t)}\\ &=\frac{q_{ij}e^{-q_{j}\cdot\Delta t}}{q_{j}-q_{i}}(e^{\Delta t(q_{j}-q_{i})}-1) \end{split} $$ with the probability of $X(t)$ remaining in state $i$ from $0$ to $\Delta t$ equal to $$1-\frac{q_{ij}e^{-q_{j}\cdot\Delta t}}{q_{j}-q_{i}}(e^{\Delta t(q_{j}-q_{i})}-1)$$.

My first question: does this approach correctly reflect the right probabilities?

These strong assumptions are made, because in this simulation only one single transition per timestep $\Delta t$ is possible. Transition times of the simulation are subsequently used to determine the likelihood, given a generator matrix; however, in determining the likelihood every transition is considered to occur at the end of every timestep $\Delta t$, which creates an error.

My second question: what approach or theory can be used to approximate the error in the aforementioned method of determining the likelihood?

Thanks in advance.

$\endgroup$
4
  • $\begingroup$ Could you explain what it might possibly mean to "exclude this effect"? The problem you formulate in the first paragraph doesn't appear soluble without adding strong assumptions about the process. For most processes in continuous time, the chance of an instantaneous transition at a specific time $\Delta t$ is zero. $\endgroup$
    – whuber
    Commented Jul 15, 2020 at 12:44
  • $\begingroup$ Using a transition probability matrix the stochastic process $X(t)$ might make transitions according to the following values $X(0)=i$, $X(\Delta t/2)=h$ and $X(X(\Delta t)=j)$ for $i\neq h\neq j$. This effect or these paths, I want to exclude and therefore want to determine the probability of a single transition from $i$ to $j$ between $0$ and $\Delta t$. I now notice that my question is slightly wrong, since it asks for the probability of a transition at exactly $\Delta t$, which I can understand has zero probability in the context of a continuous timeline. Does this comment clarify? $\endgroup$
    – user290384
    Commented Jul 15, 2020 at 13:00
  • $\begingroup$ It helps me understand the question doesn't ask what you want it to ask, so please edit it accordingly. $\endgroup$
    – whuber
    Commented Jul 15, 2020 at 13:08
  • $\begingroup$ I have edited the question and added an extra subquestion, which came to mind while writing out the problem. $\endgroup$
    – user290384
    Commented Jul 15, 2020 at 13:48

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.