2
$\begingroup$

Given a sample of small finite sequences of transitions between the states of a Markov chain with unknown probabilities how to find the probabilities of transitions between states? Would computing the naive probabilities be a right (best) method? By the naive method I mean to compute the fraction of transitions that went from a given state to one of the possible states?

$\endgroup$
2
$\begingroup$

Your intuition is correct, the empirical transition probabilities are in fact the Maximum Likelihood Estimator for the transition matrix. Note that this is not the only right/best way, but it's probably the most convenient. For example, you could cook up a prior over transition matrices and then use the likelihood given in the reference to calculate a posterior distribution over transition matrices; from there you would use the posterior mean as your "best estimate", rather than the MLE.

EDIT/ADDENDUM An example of the MLE approach is as follows: say we have a markov chain with states $S_i$. Let $p_{ij}$ denote the probability of transition between $S_i$ and $S_j$. We would like to attain the MLE estimate for $\hat{p}_{ij}$. Following the reference above it can be show that this is just $\hat{p}_{ij} = \frac{n_{ij}}{\sum\limits_{j=1}^3 n_{ij}}$ where $n_{ij}$ is the total number of observed transitions from $S_i$ to $S_j$. Note that the sum in the denominator is taken over the $j$ indices only. This sum is calculates the total number of state transitions that have departed from $S_i$.

$\endgroup$
  • $\begingroup$ Can you illustrate how this would work out in an example? $\endgroup$ – Jon Oct 11 '17 at 15:47
  • $\begingroup$ Which approach: the MLE estimator or the Posterior Mean? $\endgroup$ – Proof.by.Accident Oct 11 '17 at 17:12
  • $\begingroup$ That is exactly the type of result I was looking for. Thanks. $\endgroup$ – Vladimir Krouglov Oct 11 '17 at 18:35

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.