Difference between a markov process and a semi- markov process

Question

As the title suggest, i have a little problem grasping the main difference between markov and semi-markov processes. Anyone up for the challenge? :)

Answer 1

I will discuss only Markov processes on finite or countable state spaces. My answer is based off of this page.

Suppose $s > t_n > ... > t_1$. A Markov process is a stochastic process where the conditional distribution of $X_s$ given $X_{t_1}, X_{t_2}, ... X_{t_n}$ depends only $X_{t_n}$.

One consequence of this definition is that the time until the next jump is exponentially distributed. This property arises because only the exponential distribution satisfies the "memoryless" property $P(X_t | X_s=x) = P(X_{t-\delta s} | X_{s - \delta s}=x)$. Another consequence is that, given a transition occurs at time $s$, the distribution over the next state depends only on the state immediately before. Thus, the upcoming transition's distribution is completely described by a product of an exponential PDF (for the waiting time) and a categorical distribution (for the next state).

For semi-Markov processes, upcoming transition's distribution is described by a product of an arbitrary PDF (for the waiting time) and a categorical distribution (for the next state). The waiting time is no longer required to be exponential; in other words, the process is allowed to "remember" not only the current state, but also how long it has been in the current state. One interesting wrinkle is that, although it remembers how long the current state has lasted, that knowledge must not affect the decision about which state to enter next. This may make semi-Markov models unsuitable for some applications: if someone is getting sicker and sicker of her job, for instance, she might be more apt to take anything that comes up, and the distribution over her next state might flatten.