0
$\begingroup$

Suppose we have a Markov chain with two states A and B. This associated transition matrix is:

\begin{equation} P_{mc}= \begin{pmatrix} 0.3 & 0.7\\ 0.6 & 0.4 \end{pmatrix} \end{equation}

This matrix is empirical and computed from a set of observations:

e.g :A->B->A->A->A->B->A->A->B->B->A->B->B->A->B->A->B->A->B->B->B

My question is, After predicting a new state (B or A) how accurate is it to generate a new transition matrix P_mc based on the new sequence and is there any theoretical limits to doing this?

| cite | improve this question | |
$\endgroup$
  • $\begingroup$ I don't quite understand what the new matrix' generation is based on - are you updating the matrix based on a new observation, or the actual prediction? $\endgroup$ – jkm Feb 21 at 10:57
  • $\begingroup$ The matrix is updated using the actual prediction $\endgroup$ – nidabdella Feb 21 at 10:59
1
$\begingroup$

If you're using your current predictions to influence the estimation of your future predictions, you get a positive feedback loop and wave goodbye to any real-world distribution.

For example, an initial transition matrix $[[0.99, 0.01], [0.01, 0.99]]$ will asymptotically wind up predicting either only As or only Bs, depending on your start state.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ I'm not sure I understood what you mean by 'you get a positive feedback loop and wave goodbye to any real-world distribution'. Are you talking about a statistical distribution or parallel computing $\endgroup$ – nidabdella Feb 21 at 12:46
  • $\begingroup$ Statistical. Your transition matrix will asymptotically collapse to predicting either A->A or A->B with 100% probability regardless of the real-world probability. $\endgroup$ – jkm Feb 21 at 13:19
  • $\begingroup$ Can you tell me on what basis you concluded this? $\endgroup$ – nidabdella Feb 21 at 13:31
  • $\begingroup$ Just think through this. Let's take a string ABBBAAAA as our starting point. Empirically, your matrix is entries for A are [.67, .33], let's roll with that. Your predictor peeks at the last A, rolls the dice, and predicts another A 2/3 of the time. Your new matrix is [.80,.20]. You now look at your new A, roll for another A 80% of the time, updating the matrix to [.83, .17], and again for [.86, .14]... Getting an A reinforces the odds of producing an A and vice versa. I didn't normalize the whole matrix for compactness' sake, but the pattern holds. $\endgroup$ – jkm Feb 21 at 14:22

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.