1
$\begingroup$

I am studying some examples of balance equations for Markov models. I am presented with the following example:

$$\mathcal{P} = \begin{bmatrix} 0.2 & 0.3 & 0.5 \\ 0.1 & 0 & 0.9 \\ 0.55 & 0 & 0.45 \end{bmatrix}$$

[dropping the $i$ subscript by writing $\pi_j$ for $\pi_{ij}.]$

The balance equations are

$$\begin{align} &\pi_1 = 0.2 \pi_1 + 0.1 \pi_2 + 0.55 \pi_3 \tag{a} \\ &\pi_2 = 0.3 \pi_1 \tag{b} \\ &\pi_3 = 0.5 \pi_1 + 0.9 \pi_2 + 0.45 \pi_3 \tag{c} \end{align}$$

Since, also, $\pi_1 + \pi_2 + \pi_3 = 1$, the unique solution is

$$\pi_1 = \frac1{2.7} = 0.37037, \ \ \ \pi_2 = \frac19 = 0.11111, \ \ \ \pi_3 = \frac{1.4}{2.7} = 0.51852$$

How do we solve this for the values $\pi_1, \pi_2, \pi_3$? Is there a way to solve this using matrix computations? The difficulty here, as I see it, is that we have a constraint $\pi_1 + \pi_2 + \pi_3 = 1$ that must hold, so I'm unsure of how this is done.

I would greatly appreciate it if someone would please take the time to show this.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

We can solve linear system of equations.

Equation $(a)$ can be converted to $$(0.2-1)\pi_1 + 0.1\pi_2 + 0.55\pi_3=0\tag{a'}$$

Similarly for $b$ and $c$.

Also, with the constraint $\pi_1+\pi_2+\pi_3=1$

We have $3$ variables and $4$ constraints.

$$\pi=P^T\pi$$ $$e^T\pi=1$$

$$\begin{bmatrix} P^T-I \\ e^T\end{bmatrix}\pi =\begin{bmatrix} 0_3 \\ 1\end{bmatrix}$$

You can perform Gaussian Elimination to get the solution.

Here is the Octave solution:

octave:1> A = [-0.8, 0.1, 0.55, 0; 0.3, -1, 0, 0; 0.5,  0.9, -0.55,  0; 1, 1, 1, 1]
A =

  -0.80000   0.10000   0.55000   0.00000
   0.30000  -1.00000   0.00000   0.00000
   0.50000   0.90000  -0.55000   0.00000
   1.00000   1.00000   1.00000   1.00000

octave:2> rref(A)
ans =

   1.00000   0.00000   0.00000   0.37037
   0.00000   1.00000   0.00000   0.11111
   0.00000   0.00000   1.00000   0.51852
   0.00000   0.00000   0.00000   0.00000
Improve this answer
$\endgroup$
5
  • $\begingroup$ I'm confused about where these came from: $$\pi=P^T\pi,$$ $$e^T\pi=1,$$ $$\begin{bmatrix} P^T-I \\ e^T\end{bmatrix}\pi =\begin{bmatrix} 0_3 \\ 1\end{bmatrix}$$ And what is $e^T$ supposed to be? Is it the transpose of the constant $e$? What does it mean to have the transpose of a constant, and where did it come from? $\endgroup$
    – The Pointer
    Commented Mar 17, 2020 at 8:06
  • 1
    $\begingroup$ oops, it's my habit to use $e$ to denote the column vector of length $n$. It comes from $\pi_i$'s sums up to $1$. $\pi^T=\pi^TP$ are the balance equations. $\endgroup$
    – Siong Thye Goh
    Commented Mar 17, 2020 at 8:09
  • $\begingroup$ Hmm, I see. Where did $\begin{bmatrix} P^T-I \\ e^T\end{bmatrix}\pi =\begin{bmatrix} 0_3 \\ 1\end{bmatrix}$ come from? $\endgroup$
    – The Pointer
    Commented Mar 17, 2020 at 8:15
  • 1
    $\begingroup$ From $\pi^T=\pi^TP$, take transpose, you get $P^T\pi = \pi$, move the right hand side to left hand side and you will get $(P^T-I)\pi = 0$, now append it with $e^T\pi=1$. $\endgroup$
    – Siong Thye Goh
    Commented Mar 17, 2020 at 8:18
  • $\begingroup$ Ahh, yes, of course. Very interesting. Thank you for taking the time to clarify! $\endgroup$
    – The Pointer
    Commented Mar 17, 2020 at 8:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.