1
$\begingroup$

Section 7.2 of the book "transition probability graph" coming from the book "Introduction to Probability, 2nd Edition by Dimitri P. Bertsekas and John N. Tsitsiklis" gives some explanation of the notion of revisiting a state precise.

enter image description here

Let's apply this to this sunny rainy example (not included in that book).

Let the initial state $i$ be sunny.

Let A(i) = {sunny, rainy} be the set of states that are accessible from i.

Let the weather chain be the sequence (sunny, sunny, sunny, sunny, rainy, rainy, rainy, sunny, sunny, sunny, sunny, rainy, rainy, sunny, sunny, sunny), which has 16 elements, illustrated by this figure.

enter image description here

The first transition $(i,i_1) = (sunny, sunny)$

The last transition $(i_{n-1},j) = (sunny, sunny)$

In this particular case, is the n equal to 15, no matter the initial state is or not probabilistic?

Is n the length of the Markov chain?

$\endgroup$

1 Answer 1

3
$\begingroup$

Yes, n is just the number of steps between states taken. In your example, n is 15.

$\endgroup$
3
  • $\begingroup$ n is just the number of steps between states taken, no matter the initial state is or not probabilistic, right? $\endgroup$
    – JJJohn
    Aug 7, 2019 at 6:15
  • $\begingroup$ Yes, as I said above. $\endgroup$
    – damerdji
    Aug 12, 2019 at 23:25
  • $\begingroup$ Would you please provide an authoritative textbook or open source implementation as a reference? $\endgroup$
    – JJJohn
    Aug 13, 2019 at 15:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.