0
$\begingroup$

Can I have a Markov chain where one state doesn't transition to any other state? (but some other states transition into it)

$\endgroup$
  • 1
    $\begingroup$ Yes. Therefore column sums of a transition matrix do not have to be 1. $\endgroup$ – Richard Hardy Feb 16 '15 at 7:09
  • $\begingroup$ Yes, in the stated example of absorbing state $i$ say, the $(i,i)$ entry would be $1$. Often it is mathematically convenient to consider "sub-stochastic" matrices (where the row sums are $<1$). Also, some people think of $<1$ rows sums as signifying the Markov chain can die in that state (which is the same as sending it to an absorbing "cemetery" state not formally included in the state space). $\endgroup$ – P.Windridge Feb 16 '15 at 13:39
  • $\begingroup$ Consider why the following might be a problem: "There's a 5% chance I'll eat lunch by 12:15 and a 80% chance I won't" (if you said "But what about the other 15%?", ... that's why you want the rows to sum to 1) $\endgroup$ – Glen_b Feb 17 '15 at 5:06
2
$\begingroup$

People often consider square matrices with non-negative entries and row sums $\leq 1$ in the context of Markov chains. They are called sub-stochastic. The usual convention is the missing mass $1- \sum [$entries in row $i]$ corresponds to the probability that the Markov chain is "killed" and sent to an imaginary absorbing "cemetery" state, when it is state $i$.

Sub-stochastic matrices often crop up in Markov chain calculations e.g. when we consider only states in a communicating class, or just transitive states etc.

$\endgroup$
1
$\begingroup$

Row sum of transition probability matrix need to be 1 because the states we define should be exhaustive.

Example: There should not be a customer who are not in any of the states defined for the markov model.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.