Timeline for re-estimation of emission probabilities in HMM
Current License: CC BY-SA 3.0
15 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| May 1, 2017 at 15:16 | history | bounty ended | CommunityBot | ||
| Apr 25, 2017 at 13:13 | comment | added | lo tolmencre | Thanks for your help! I got it working now, but with the formula for inititial probabilities that actally involves the $t = 1$ part. Again, thanks a lot! | |
| Apr 24, 2017 at 16:30 | comment | added | lo tolmencre | which would in turn be the same as \begin{align*} &\forall i : \text{HMM.init-prob(i)} & \leftarrow &&\frac{\text{expect-state}[i, 1]}{\sum_{j=1}^N \text{expect-state}[j, 1]}\end{align*} | |
| Apr 24, 2017 at 16:28 | comment | added | lo tolmencre | Ah ok, but then isn't the re-estimate of the initial probabilities somehow lacking information specific to initial probabilities? The $t = 1$ part is comepletly absent from that re-estimate: \begin{align*} &\forall i : \text{HMM.init-prob(i)} & \leftarrow &&\frac{\text{expect-state}[i]}{\sum_{j=1}^N \text{expect-state}[j]}\\ \end{align*} Isn't that different from $\pi_i^* = \frac{\sum_{y \in Y} \gamma_1(i)}{\sum_{y \in Y} \sum_{j=1}^N \gamma_1(j)}$? | |
| Apr 24, 2017 at 16:12 | comment | added | jkpate | The first one is almost right, except for the emission probability. The denominator should be only the expected count of state $i$: $\forall i, e : \text{HMM.emiss-prob(i, e)} \leftarrow \frac{\text{expect-emiss}[i, e]} {\text{expect-state}[i]}$ | |
| Apr 24, 2017 at 11:22 | vote | accept | lo tolmencre | ||
| Apr 24, 2017 at 11:22 | comment | added | lo tolmencre | Thanks. I made a third edit to my post to verify if I got this right now, could you have a look at it? I would really appreciate it. | |
| Apr 23, 2017 at 19:38 | comment | added | jkpate | Yes, you're right. The example on Wikipedia looks like it's designed with only one sequence in mind. | |
| Apr 23, 2017 at 18:37 | comment | added | lo tolmencre | Ok, so then this means that for instance the re-estimated initial probability of state $i$ is $\pi_i^* = \frac{\sum_{y \in Y} \gamma_1(i)}{\sum_{y \in Y} \sum_{j=1}^N \gamma_1(j)}$ rather than $\pi_i^* = \gamma_1(i)$ as it says for example on Wiki? ($Y$ being tha data set)(en.wikipedia.org/wiki/Baum%E2%80%93Welch_algorithm#Update) Or maybe the notation in the wiki article means exactly the same? | |
| Apr 23, 2017 at 18:03 | comment | added | jkpate | Just sum the expected counts. Your total expected count for $s_k$ emitting $e_l$ is the expected count for sequence one plus the expected count for sequence 2 and so on, all the way through the dataset. | |
| Apr 23, 2017 at 16:24 | comment | added | lo tolmencre | Ah ok, that never got stated explictely in all sources I considered :/ However, what do I do with the expected counts that I already have? Do I overwrite them or do I store all of them and in the end compute their respective average? | |
| Apr 23, 2017 at 15:42 | comment | added | jkpate | I saw in another question of yours that you believe that transition probabilities are updated after every training sequence. This is not true for vanilla EM algorithms like Baum-Welch. For vanilla EM, all model parameters are updated only after gathering expected counts from the entire dataset. For HMMs, you gather expected counts for emissions and transition probabilities from the entire dataset, and only then do you re-estimate $\mathbf{a}$ and $\mathbf{b}$. If you want to update earlier, then you'll need to do something more sophisticated like the incremental EM algorithm I described. | |
| Apr 23, 2017 at 15:21 | history | edited | jkpate | CC BY-SA 3.0 |
deleted 4 characters in body
|
| Apr 23, 2017 at 15:12 | history | edited | jkpate | CC BY-SA 3.0 |
added 6 characters in body
|
| Apr 23, 2017 at 15:05 | history | answered | jkpate | CC BY-SA 3.0 |