# Learning HMM parameters by counting?

In 8.4.3 of the book Speech and Language Processing: An introduction to natural language processing, the two matrices transition probabilities and emission probabilities can be learned by counting as follows:

$$P(t_i|t_{i-1})=\frac{C(t_{i-1}, t_i)}{C(t_{i-1})}$$ $$P(w_i|t_i)=\frac{C(t_i, w_i)}{C(t_i)}$$ where $$t_i$$ is a state and $$w_i$$ is a word.

But I remember that it is the EM algorithm that should be applied. Am I missing something?

• This is the fully observed case, for when you have the tag sequence (the latent variables) provided. EM a is used for when your data is “partially observed”: you don’t have the values of the latent variables. All you have is the text, not the tags. Sep 15 at 12:50
• @AryaMcCarthy If it's unsupervised, how do we know which state represents which meaning? For instance, we only know there should be five states(A, B, C, D, and E), but we don't know which state stands for A. Sep 15 at 13:25
• If you’ve never observed the states, then you don’t ever know. The best you can do is to try some post-hoc matching. There’s a decently famous paper about what happens to the hidden states for text by Merialdo (1994). (But remember that J&M do observe the states! That lets them steer the parameters however they want.) Sep 15 at 15:50

1. A base level of states ($$t$$) which transition from one to the next.
2. An emitted level of states ($$w$$) whose observation depends on the base states.
In a true HMM, the states $$t$$ are hidden and you have to use EM to get a maximal likelihood estimation of what they might be.
In this case, you actually (if I've understood the text correctly) have both the $$t$$ and $$w$$ states as observations so you can compute the conditional probabilities directly by just counting.