1
$\begingroup$

I am studying some examples of balance equations for Markov models. I am presented with the following example:

$$\mathcal{P} = \begin{bmatrix} 0.2 & 0.3 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.55 & 0 & 0.45 \end{bmatrix}$$

[dropping the $i$ subscript by writing $\pi_j$ for $\pi_{ij}.]$

The balance equations are

$$\begin{align} &\pi_1 = 0.2 \pi_1 + 0.1 \pi_2 + 0.55 \pi_3 \tag{a} \\ &\pi_2 = 0.3 \pi_1 \tag{b} \\ &\pi_3 = 0.5 \pi_1 + 0.9 \pi_2 + 0.45 \pi_3 \tag{c} \end{align}$$

Since, also, $\pi_1 + \pi_2 + \pi_3 = 1$, the unique solution is

$$\pi_1 = \frac1{2.7} = 0.37037, \ \ \ \pi_2 = \frac19 = 0.11111, \ \ \ \pi_3 = \frac{1.4}{2.7} = 0.51852$$

How do we solve this for the values $\pi_1, \pi_2, \pi_3$? Is there a way to solve this using matrix computations? The difficulty here, as I see it, is that we have a constraint $\pi_1 + \pi_2 + \pi_3 = 1$ that must hold, so I'm unsure of how this is done.

I would greatly appreciate it if someone would please take the time to show this.

$\endgroup$
2
$\begingroup$

We can solve linear system of equations.

Equation $(a)$ can be converted to $$(0.2-1)\pi_1 + 0.1\pi_2 + 0.55\pi_3=0\tag{a'}$$

Similarly for $b$ and $c$.

Also, with the constraint $\pi_1+\pi_2+\pi_3=1$

We have $3$ variables and $4$ constraints.

$$\pi=P^T\pi$$ $$e^T\pi=1$$

$$\begin{bmatrix} P^T-I \\ e^T\end{bmatrix}\pi =\begin{bmatrix} 0_3 \\ 1\end{bmatrix}$$

You can perform Gaussian Elimination to get the solution.

Here is the Octave solution:

octave:1> A = [-0.8, 0.1, 0.55, 0; 0.3, -1, 0, 0; 0.5,  0.9, -0.55,  0; 1, 1, 1, 1]
A =

  -0.80000   0.10000   0.55000   0.00000
   0.30000  -1.00000   0.00000   0.00000
   0.50000   0.90000  -0.55000   0.00000
   1.00000   1.00000   1.00000   1.00000

octave:2> rref(A)
ans =

   1.00000   0.00000   0.00000   0.37037
   0.00000   1.00000   0.00000   0.11111
   0.00000   0.00000   1.00000   0.51852
   0.00000   0.00000   0.00000   0.00000
| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I'm confused about where these came from: $$\pi=P^T\pi,$$ $$e^T\pi=1,$$ $$\begin{bmatrix} P^T-I \\ e^T\end{bmatrix}\pi =\begin{bmatrix} 0_3 \\ 1\end{bmatrix}$$ And what is $e^T$ supposed to be? Is it the transpose of the constant $e$? What does it mean to have the transpose of a constant, and where did it come from? $\endgroup$ – The Pointer Mar 17 at 8:06
  • 1
    $\begingroup$ oops, it's my habit to use $e$ to denote the column vector of length $n$. It comes from $\pi_i$'s sums up to $1$. $\pi^T=\pi^TP$ are the balance equations. $\endgroup$ – Siong Thye Goh Mar 17 at 8:09
  • $\begingroup$ Hmm, I see. Where did $\begin{bmatrix} P^T-I \\ e^T\end{bmatrix}\pi =\begin{bmatrix} 0_3 \\ 1\end{bmatrix}$ come from? $\endgroup$ – The Pointer Mar 17 at 8:15
  • 1
    $\begingroup$ From $\pi^T=\pi^TP$, take transpose, you get $P^T\pi = \pi$, move the right hand side to left hand side and you will get $(P^T-I)\pi = 0$, now append it with $e^T\pi=1$. $\endgroup$ – Siong Thye Goh Mar 17 at 8:18
  • $\begingroup$ Ahh, yes, of course. Very interesting. Thank you for taking the time to clarify! $\endgroup$ – The Pointer Mar 17 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.