$\newcommand{\E}{\mathrm{E}}$ I don't understand why Baum-Welch algorithm is an instantiation of EM algorithm.

Indeed, why computing $\alpha_t(i)$ and $\beta_t(i)$ corresponds to Expectation step. Expectation step corresponds to compute expectation over the latent variable of log-likelihood of observed variable given the parameter: $\E_{Z|X,\theta^{t-1}}[L(X|\theta)]$.

I don't understand the link.


The EM algorithm computes the expectation of the complete data log-likelihood, not the observed data log-likelihood, conditional on the observed data and current estimate of the parameters. The complete data log-likelihood can be written with indicators for the latent data. The expectation of an indicator is its probability. The expectation of the product of two indicators is the joint probability.

$\gamma_i(t) = E[ I(X_t=i) \vert Y, \theta] = P(X_t=i \vert Y, \theta)$

$\xi_i(t) = E[ I(X_t=i)I(X_{t+1}=j) \vert Y, \theta] = P(X_t=i, X_{t+1}=j \vert Y, \theta)$

Computing each of these conditional probabilities is the E-step, but computing them requires computing each of them requires computing all the $\alpha$'s and $\beta$'s.

See the Gaussian mixture example for a simpler illustration of how latent variables can be written as indicators in the complete data likelihood http://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.