1
$\begingroup$

Let $\{X_t\}_t$ be a discrete-time Markov chain with right stochastic transition matrix $P$ and a unique invariant distribution $\pi$. Let the state space be $\{1,\dots,S\}$. Is there an explicit solution (potentially an approximation) for the element $\pi^{(s)}$ of $\pi$ in terms of the elements of $P$ for a given $S$?

For instance, if \begin{equation} P = \begin{bmatrix} 1-\alpha, \alpha \\ \beta, 1-\beta \end{bmatrix} \end{equation} then \begin{equation} \pi = \begin{bmatrix} \frac{\beta}{\alpha+\beta} \\ \frac{\alpha}{\alpha+\beta} \end{bmatrix}, \end{equation} and I am interested in the generalization for $S > 2$.

$\endgroup$
1
$\begingroup$

For ergodic $\mathbf{P},$ you seek $S$-vector$\pi$ with $\pi\mathbf{P}=\pi.$ So $\pi$ is a left eigen-vector of $\mathbf{P}.$ You can use R to compute $\pi.$

Illustration:

P = matrix(c(.1, .2, .7,
             .2, .7, .1,
             .7, .1, .2), byrow=T, nrow=3)

P
     [,1] [,2] [,3]
[1,]  0.1  0.2  0.7
[2,]  0.2  0.7  0.1
[3,]  0.7  0.1  0.2

g = eigen(t(P))$vec[,1]
g = as.numeric(g)
pi = g/sum(g);  pi
[1] 0.3333333 0.3333333 0.3333333

pi %*% P
          [,1]      [,2]      [,3]
[1,] 0.3333333 0.3333333 0.3333333

I used a doubly-stochastic matrix so the answer would be obvious, but the method works generally.

The invariant (stationary) vector is proportional to the left eigen vector of smallest modulus.

Notes on R code:

(a) Use transpose t(P) because R finds right eigen-vectors.

(b) Use $vec[,1] to get the first column of the display of eigen vectors; R prints the eigen-vector of smallest modulus first.

(c) Use as.numeric to suppress (possible) complex-number notation. [For an ergodic $\mathsf{P},$ the first eigen-vector is always real, but other vectors in the display may be complex.]

(d) Use pi = g/sum(g) to make the elements of $\pi$ sum to $1.$

(e) The last line is to verify that $\pi$ is the desired vector. (Unnecessary, but guards against typos in the code.)

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.