0
$\begingroup$

I have a beginners question on Markov chains. We found that a Markov chain might be a good way to describe the data we got from our experiment. Let's say we have two simple Markov transition matrix with 3 states.

Markov transition probability matrix 1
   1     2    3
1 0.00 1.00 0.00
2 0.50 0.00 0.50
3 0.00 0.66 0.33

Markov transition probability matrix 2    
   1     2    3
1 0.66 0.33 0.00
2 0.50 0.50 0.00
3 0.00 1.00 0.00

I wonder what would be the best way to report it in an article, e.g. APA. Ideally we would like to look at differences between two different Markov transition matrices. Links to good beginner's introductions would help too. Many thanks!

What would be the most concise way to report the difference between two matrices?

$\endgroup$
1
$\begingroup$

Do you have a predefined start state ?

It is fairly easy to see that in matrix 2, state 3 has no incoming edges. And thus over time, the probability of being in state 3 converges to 0. And if the starting state is not 3, then it is exactly 0.

I think it will be interesting to report the stationary distribution of matrix 1.

One way of comparing Markov proccesses

Usually when dealing with Markov chains, you have a transition matrix $T$ and a reward/cost for being in each state $\vec{r}$

If the stationary distribution $\vec{\pi}$ exists for the two Markov processes, then you have two expected rewards: $$E_1=\vec{r}\vec{\pi_1}$$ $$E_2=\vec{r}\vec{\pi_2}$$ And you can compare which process is more rewarding / less costly

Loosely speaking, A stationary distribution is the probability of being in each state when repeating the transitions for infinity.

$\endgroup$
  • $\begingroup$ Thanks your explanation! It's very much appreciated. What about some sort of inferential statistics? My real question would be, are they different enough to be regarded from different samples? The dummy data messed the distribution up a bit, there is better data before edit. To answer, the current matrix does always start with state 1, but I'm not sure if that will truly help with comparison. Thanks again, I'll read up on the stationary distribution in the coming days, but I guess to interpret the results, we don't really want to run it to infinity (the speed of transition would also matter). $\endgroup$ – puslet88 Sep 12 '15 at 12:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.