1
$\begingroup$

I am currently studying the textbook Introduction to Modeling and Analysis of Stochastic Systems, Second Edition, by V. G. Kulkarni. In a section on discrete-time Markov chains, the author introduces the one-step transition probability matrix as follows:

$$P = \begin{bmatrix} p_{1, 1} & p_{1, 2} & p_{1, 3} & \dots & p_{1, N} \\ p_{2, 1} & p_{2, 2} & p_{2, 3} & \dots & p_{2, N} \\ p_{3, 1} & p_{3, 2} & p_{3, 3} & \dots & p_{3, N} \\ \vdots & \vdots & \vdots & \ddots & \vdots \\ p_{N, 1} & p_{N, 2} & p_{N, 3} & \dots & p_{N, N} \end{bmatrix} \tag{2.4}$$

The matrix $P$ in the equation above is called the one-step transition probability matrix, or transition matrix for short, of the DTMC. Note that the rows correspond to the starting state and the columns correspond to the ending state of a transition.

I'm not sure what the author means by the following:

... the rows correspond to the starting state and the columns correspond to the ending state of a transition.

I would greatly appreciate it if people would please take the time to clarify this.

$\endgroup$
4
  • $\begingroup$ Please see stats.stackexchange.com/search?q=markov+transition+matrix for links to many examples here on CV. Focusing on upvoted posts with answers, I found a good (albeit brief) answer at stats.stackexchange.com/a/246865/919. Following that in the list are many clear, simple examples. $\endgroup$
    – whuber
    Jan 23 '20 at 19:47
  • $\begingroup$ @whuber eh, are you sure this answers my question? My question seems relatively simple, and I don’t see how that answer addresses it. $\endgroup$ Jan 23 '20 at 19:50
  • $\begingroup$ I haven't claimed it does: I am asking whether the top hits in those searches answer the question (well, maybe even suggesting they ought to). I do hope that by reviewing them you will get closer to understanding this concept. $\endgroup$
    – whuber
    Jan 23 '20 at 19:54
  • $\begingroup$ @whuber oh, ok. I’ve looked through several of them, and they’re all relatively complex questions and answers compared to the very simple question I’m asking here. $\endgroup$ Jan 23 '20 at 19:57
1
$\begingroup$

They're just saying that the probability of ending in state $j$, given that you start in state $i$ is the element in the $i$th row and $j$th column of the matrix. For example, if you start in state $3$, the probability of transitioning to state $7$ is the element in the 3rd row, and 7th column of the matrix: $p_{37}$.

$\endgroup$
1
  • 1
    $\begingroup$ Ahh, that makes sense. Thank you for the clarification. $\endgroup$ Jan 23 '20 at 20:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.