0
$\begingroup$

I am a bit confused as I need to calculate the Markov-Chain transition probabilites for 3 variables.

Example data, let's assume a sequence of letters at specific and progressively-constant time steps:

Q
Q
E
Q
C
C
E

What are my transitional probabilities?

My (wrong) understanding is:

P(Q|Q) = 1
P(Q|E) = 1
P(Q|C) = 0

P(E|E) = 0
P(E|C) = 1
P(E|Q) = 1

P(C|C) = 1
P(C|E) = 0
P(C|Q) = 1

And therefore my (wrong) transition matrix will be:

   Q   E   C
Q  1   1   1  
E  1   0   0
C  0   1   1

note row sums are not = 1

What am I missing? The same approach works with 2 variables and here it seems that I need to divide each row by the number of probabilities > 0 to make the row sums =1.

Thanks

$\endgroup$
2
$\begingroup$

I'll give one example and hopefully it will be evident how it can be applied to the rest:

$P(X(t) = E | X(t-1) = Q)$ can be estimated empirically as "the percentage of instances of Q that are followed by E". So in this case, there are 3 instances of Q, one of which is followed by an instance of E, meaning that $P(X(t) = E | X(t-1) = Q)$ is equal to 1/3.

Side note: I think when you write P(E|Q), you mean the expression I wrote above, but it's probably better to write it how I did just to be clear that you mean "the probability of E coming directly after Q".

$\endgroup$
1
$\begingroup$

In any Transition Probability Matrix, the row sum must be equal to 1. Your Probabilities are incorrect. It should be

P(Q|Q) = 1/3
p(Q|E) = 1
P(Q|C) = 0

P(E|E) = 0
P(E|C) = 1/2
P(E|Q) = 1/3

P(C|C) = 1/2
P(C|E) = 0
P(C|Q) = 1/3

Hence the Transition Probability Matrix becomes :

   Q    E    C
Q  1/3  1/3  1/3  
E  1    0    0
C  0    1/2  1/2
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.