Suppose a Markov chain with two discrete states $A$ and $B$. The probability of moving from $A$ to $B$ is $0.1$ and the probability of moving from $A$ to $A$ is $0.9$. Similarly, $B$ to $B$ has probability $0.9$ and $B$ to $A$ probability $0.1$.

Suppose the Markov chain describes a time-series where transitions from $A$ to $B$ and from $B$ to $A$ are something one would like to predict.

Given the current state, the best prediction of the next state is the most probable next state. With the described Markov chain, it would be the best to predict $A$ or $B$ constantly, depending on the where the process started.

This fails to be useful in predicting occurrences from $A$ to $B$ or vice versa, which is interesting. What should I conclude? That making predictions in such system is not possible? Or is the reasoning flawed with respect to

best prediction of the next state is the most probable next state

I asked the question https://stats.stackexchange.com/questions/168892/how-are-markov-chains-used-for-time-series-forecasting previously. This is related to that but not equal. Are Markov chain always not good for time-series prediction?

  • $\begingroup$ Could you explain in what sense you think "this (prediction of the next state) fails to be useful" or is "always not good" when in fact it will be correct 90% of the time? $\endgroup$ – whuber Aug 26 '15 at 22:03
  • 1
    $\begingroup$ It's not at all clear what you mean by failing to be useful. If you flip a coin, is it not useful to say that heads will occur with probability $0.5$? If your system is a memoryless like a Markov chain then the only thing you can say about the next state is its probability, given the current state. The "most probable state" will be a terrible prediction if the transition probabilities are close to uniform, i.e. high entropy, like in the example of a coin. In the case of a coin, there's essentially nothing you can do to make more accurate predictions. $\endgroup$ – Alex R. Aug 27 '15 at 1:57
  • $\begingroup$ I think it is clear that this prediction is not helpful if your interested in being able to predict when A switches to B and vice versa. In this case you might want to extend your model family and consider models "with memory," where transition probabilities depend on more than the current state. For instance, you could check whether the number of As or Bs in a row follow the distribution implied by the memory less assumption (geometric, right?). $\endgroup$ – jlimahaverford Aug 27 '15 at 3:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.