# Computation of balance equation example in Markov model

I am studying some examples of balance equations for Markov models. I am presented with the following example:

$$\mathcal{P} = \begin{bmatrix} 0.2 & 0.3 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.55 & 0 & 0.45 \end{bmatrix}$$

[dropping the $$i$$ subscript by writing $$\pi_j$$ for $$\pi_{ij}.]$$

The balance equations are

\begin{align} &\pi_1 = 0.2 \pi_1 + 0.1 \pi_2 + 0.55 \pi_3 \tag{a} \\ &\pi_2 = 0.3 \pi_1 \tag{b} \\ &\pi_3 = 0.5 \pi_1 + 0.9 \pi_2 + 0.45 \pi_3 \tag{c} \end{align}

Since, also, $$\pi_1 + \pi_2 + \pi_3 = 1$$, the unique solution is

$$\pi_1 = \frac1{2.7} = 0.37037, \ \ \ \pi_2 = \frac19 = 0.11111, \ \ \ \pi_3 = \frac{1.4}{2.7} = 0.51852$$

How do we solve this for the values $$\pi_1, \pi_2, \pi_3$$? Is there a way to solve this using matrix computations? The difficulty here, as I see it, is that we have a constraint $$\pi_1 + \pi_2 + \pi_3 = 1$$ that must hold, so I'm unsure of how this is done.

I would greatly appreciate it if someone would please take the time to show this.

We can solve linear system of equations.

Equation $$(a)$$ can be converted to $$(0.2-1)\pi_1 + 0.1\pi_2 + 0.55\pi_3=0\tag{a'}$$

Similarly for $$b$$ and $$c$$.

Also, with the constraint $$\pi_1+\pi_2+\pi_3=1$$

We have $$3$$ variables and $$4$$ constraints.

$$\pi=P^T\pi$$ $$e^T\pi=1$$

$$\begin{bmatrix} P^T-I \\ e^T\end{bmatrix}\pi =\begin{bmatrix} 0_3 \\ 1\end{bmatrix}$$

You can perform Gaussian Elimination to get the solution.

Here is the Octave solution:

octave:1> A = [-0.8, 0.1, 0.55, 0; 0.3, -1, 0, 0; 0.5,  0.9, -0.55,  0; 1, 1, 1, 1]
A =

-0.80000   0.10000   0.55000   0.00000
0.30000  -1.00000   0.00000   0.00000
0.50000   0.90000  -0.55000   0.00000
1.00000   1.00000   1.00000   1.00000

octave:2> rref(A)
ans =

1.00000   0.00000   0.00000   0.37037
0.00000   1.00000   0.00000   0.11111
0.00000   0.00000   1.00000   0.51852
0.00000   0.00000   0.00000   0.00000

• I'm confused about where these came from: $$\pi=P^T\pi,$$ $$e^T\pi=1,$$ $$\begin{bmatrix} P^T-I \\ e^T\end{bmatrix}\pi =\begin{bmatrix} 0_3 \\ 1\end{bmatrix}$$ And what is $e^T$ supposed to be? Is it the transpose of the constant $e$? What does it mean to have the transpose of a constant, and where did it come from? Mar 17, 2020 at 8:06
• oops, it's my habit to use $e$ to denote the column vector of length $n$. It comes from $\pi_i$'s sums up to $1$. $\pi^T=\pi^TP$ are the balance equations. Mar 17, 2020 at 8:09
• Hmm, I see. Where did $\begin{bmatrix} P^T-I \\ e^T\end{bmatrix}\pi =\begin{bmatrix} 0_3 \\ 1\end{bmatrix}$ come from? Mar 17, 2020 at 8:15
• From $\pi^T=\pi^TP$, take transpose, you get $P^T\pi = \pi$, move the right hand side to left hand side and you will get $(P^T-I)\pi = 0$, now append it with $e^T\pi=1$. Mar 17, 2020 at 8:18
• Ahh, yes, of course. Very interesting. Thank you for taking the time to clarify! Mar 17, 2020 at 8:18