As the title suggest, i have a little problem grasping the main difference between markov and semi-markov processes. Anyone up for the challenge? :)


1 Answer 1


I will discuss only Markov processes on finite or countable state spaces. My answer is based off of this page.

Suppose $s > t_n > ... > t_1$. A Markov process is a stochastic process where the conditional distribution of $X_s$ given $X_{t_1}, X_{t_2}, ... X_{t_n}$ depends only $X_{t_n}$.

One consequence of this definition is that the time until the next jump is exponentially distributed. This property arises because only the exponential distribution satisfies the "memoryless" property $P(X_t | X_s=x) = P(X_{t-\delta s} | X_{s - \delta s}=x)$. Another consequence is that, given a transition occurs at time $s$, the distribution over the next state depends only on the state immediately before. Thus, the upcoming transition's distribution is completely described by a product of an exponential PDF (for the waiting time) and a categorical distribution (for the next state).

For semi-Markov processes, upcoming transition's distribution is described by a product of an arbitrary PDF (for the waiting time) and a categorical distribution (for the next state). The waiting time is no longer required to be exponential; in other words, the process is allowed to "remember" not only the current state, but also how long it has been in the current state. One interesting wrinkle is that, although it remembers how long the current state has lasted, that knowledge must not affect the decision about which state to enter next. This may make semi-Markov models unsuitable for some applications: if someone is getting sicker and sicker of her job, for instance, she might be more apt to take anything that comes up, and the distribution over her next state might flatten.

  • $\begingroup$ Thanks! That was actually a really good way of explaining it, thank you :) $\endgroup$
    – Janono
    Dec 12, 2017 at 21:23
  • $\begingroup$ Glad I could help! I learned something new, too. $\endgroup$ Dec 12, 2017 at 21:36
  • $\begingroup$ @ Eric_kernfeld: great answer! I posted a similar question here - can you please take a look at it? stats.stackexchange.com/questions/585287/… Thank you so much! $\endgroup$
    – stats_noob
    Nov 30, 2022 at 3:20
  • $\begingroup$ I will try to take a look this weekend, but no guarantees. Sorry if I am not able to. $\endgroup$ Dec 2, 2022 at 21:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.