How to report simple markov transition matrix

Question

I have a beginners question on Markov chains. We found that a Markov chain might be a good way to describe the data we got from our experiment. Let's say we have two simple Markov transition matrix with 3 states.

Markov transition probability matrix 1
   1     2    3
1 0.00 1.00 0.00
2 0.50 0.00 0.50
3 0.00 0.66 0.33

Markov transition probability matrix 2    
   1     2    3
1 0.66 0.33 0.00
2 0.50 0.50 0.00
3 0.00 1.00 0.00

I wonder what would be the best way to report it in an article, e.g. APA. Ideally we would like to look at differences between two different Markov transition matrices. Links to good beginner's introductions would help too. Many thanks!

What would be the most concise way to report the difference between two matrices?

Answer 1

Do you have a predefined start state ?

It is fairly easy to see that in matrix 2, state 3 has no incoming edges. And thus over time, the probability of being in state 3 converges to 0. And if the starting state is not 3, then it is exactly 0.

I think it will be interesting to report the stationary distribution of matrix 1.

One way of comparing Markov proccesses

Usually when dealing with Markov chains, you have a transition matrix $T$ and a reward/cost for being in each state $\vec{r}$

If the stationary distribution $\vec{\pi}$ exists for the two Markov processes, then you have two expected rewards: $$E_1=\vec{r}\vec{\pi_1}$$ $$E_2=\vec{r}\vec{\pi_2}$$ And you can compare which process is more rewarding / less costly

Loosely speaking, A stationary distribution is the probability of being in each state when repeating the transitions for infinity.