2
$\begingroup$

Summary: The basic goal is to find the time evolution of the probability generating function (or the moment generating function or the characteristic function if you prefer) for a continuous-time markov chain with a finite number of states.

To set notation for a particular time period: let $X$ be a random variable with a discrete number of states $n = 1,\ldots N$. Let the probability distribution of this be $\mathbb{P}_n \equiv Pr(X = n)$, with the usual properties such as $\sum_{n=1}^N \mathbb{P}_n = 1$. Furthermore, use the standard definition of the probability generating function of this random variable as $$ G(z) \equiv \sum_{n=1}^{N}\mathbb{P}_n z^n $$ (Note that if we summed to infinity, it would be equivalent as it only has support up to $N$). Finally, summarize the probability distribution by the vector $\mathbb{P} \equiv \{\mathbb{P}_1, \mathbb{P}_2,\ldots \mathbb{P}_N\}\in\mathbb{R}^N$. Note that typically the conversion between $\mathbb{P}$ and $G(z)$ is done with a z-transform.

Stochastic process for $X$: In reality, the random variable $X$ evolves as a continuous time stochastic process $X_t$ for $t\geq 0$. The process is a continuous-time markov chain (http://en.wikipedia.org/wiki/Continuous-time_Markov_chain) with some infinitesimal generator $\mathbb{Q} \in \mathbb{R}^{N\times N}$

Denote $\mathbb{P}_n(t) \equiv Pr(X_t = n)$, stacking the vector of probabilities at any particular point in time as $\mathbb{P}(t)$. With this, given some initial condition of the probability distribution $\mathbb{P}(0)$, the evolution of the probability distribution is given by solving the Kolmogorov Forward Equation as a system of $N$ ODEs, $$ \frac{d\mathbb{P}(t)}{d t} = \mathbb{P}(t)\cdot \mathbb{Q},\,\text{subject to }\mathbb{P}(0) $$ Defining with matrix exponentials, this ODE has the general solution of $$\mathbb{P}(t) = e^{t\mathbb{Q}}$$

Of course, I could denote the probability generating function of $X_t$ as $$ G(t, z) \equiv \sum_{n=1}^{\infty}\mathbb{P}_n(t) z^n $$

Just to be clear: The typical properties of the probability generating function, $G(t,z)$, only apply to the non-time dimensions, i.e. $z$.


Up until now, everything is completely standard and follows the usual definitions. My basic question is whether we can write an expression for the evolution of $G(t,z)$ either directly, or as a partial differential equation that I could solve numerically. Why would this be useful? Because $\frac{d^K G(t,1)}{ d z^K}$ are the factorial moments of the random variable $X_t$ for a fixed time $t$. See http://en.wikipedia.org/wiki/Probability-generating_function#Probabilities_and_expectations

Version 1 of Question: From $\mathbb{Q}$ is there a way to write a differential equation for $G(t,z)$ directly?

Version 2 of Question: is there any clean way to write/simplify $G(t,z)$ in terms of $e^{t \mathbb{Q}}$ (other than the obvious, direct unsimplified substitution into the definition)? Feel free to use an eigen-decomposition $\mathbb{Q} \equiv U D U^{-1}$ so that $e^{t \mathbb{Q}} = U e^{t D}U^{-1}$ for some diagonal $D$

As a simple example: the telegraph process with switching intensity $\lambda > 0$ has $N=2$ and infinitesimal generator $\mathbb{Q} = \begin{bmatrix}-\lambda & \lambda\\ \lambda & - \lambda\end{bmatrix} $. If we started in state $1$ with certainty, i.e., $\mathbb{P}_1(0) = 1$, then with a $G(t,z)$ for this $\mathbb{Q}$, we could find the evolution of the moments over time by taking the derivatives of $G(t,z)$ at $z=1$. Also, see http://www.columbia.edu/~ww2040/TransientMM1questa.pdf for an example of this with a particular stochastic process (albeit one with an infinite support for $n$).


One last point: if you prefer to work with the characteristic function, the moment generating function, or anything else with an equivalence to the the probability distribution function, that is fine.

$\endgroup$
  • $\begingroup$ Why stop the series defining $G$ at $n=N$ the number of states? What is $\mathbb P_n$? What are your thoughts? $\endgroup$ – Did Feb 23 '15 at 7:34
  • $\begingroup$ @Did Thanks for responding. My general problems have a a finite number of states, $N$. As the probability of $n > N$ is $0$, doesn't this stop at $N$? I guess it doesn't hurt to write this as infinite. $\endgroup$ – jlperla Feb 23 '15 at 18:15
  • $\begingroup$ $\mathbb{P}_n(t)$ is the probability of the random variable being $n$ at time $t$. $\mathbb{P}(t)$ is the vector of all probabilities. $\endgroup$ – jlperla Feb 23 '15 at 18:16
  • $\begingroup$ There's an inconsistency: the MC is declared to be over $N$ states but the generating function explicitly assumes the states are the natural numbers $1, 2, \ldots, n, \ldots$. Please edit the question to clear this up. Note that if $G$ really is a sum over the states, then it's just another way of writing down $\mathbb{P}(t)$, which seems like a rather trivial (and pointless) change of notation. What is the purpose behind this? $\endgroup$ – whuber Feb 23 '15 at 20:02
  • $\begingroup$ @whuber Sorry if I have made an obvious mistake... As an example of the moment generating function for a finite state probability distribution, see this derivation for the binomial distribution: le.ac.uk/users/dsgp1/COURSES/MATHSTAT/5binomgf.pdf I changed it to sum to infinity, but as the probability distribution wouldn't have support above $N$, I don't think it matters. $\endgroup$ – jlperla Feb 23 '15 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.