1
$\begingroup$

In the HMM formulation where z is hidden state and x is observed

In the forward case, I see it represented as such:

$\alpha_{k}(z_{k})=P(z_{k},x_{1:k})=\sum_{z_{k-1}}P(z_{k},z_{k-1},x_{1:k})$

but in the backward case I see it as such

$\beta_{k}(z_{k})=P(x_{k+1:n}|z_{k})=\sum_{z_{k+1}}P(x_{k+1:n},z_{k+1}|z_{k})$


I thought that in the probability rules, we can factor in a new term based on this expression:

$P(A,B)=\sum_{C}P(A,C|B)$

If so, how are both of those expressions even expanded? Shoudn't the forward rule then be:

$\alpha_{k}(z_{k})=P(z_{k},x_{1:k})=\sum_{z_{k-1}}P(z_{k},z_{k-1}|x_{1:k})$

and the backward rule be not possible to factorize based on the above rule? Is my rule wrong?

$\endgroup$
2
  • $\begingroup$ The rule is not correct. Could you provide more information on why you think it is a valid rule so I can provide a better answer? $\endgroup$
    – jkpate
    Dec 1 '18 at 22:42
  • $\begingroup$ Then can I know the correct rule please? I need to know the general rule that allows for the formulation of those two backward forward expressions. $\endgroup$
    – raaj
    Dec 1 '18 at 22:48
2
$\begingroup$

The probability rule is not right. I think you are thinking of the sum rule: \begin{align} P( A ) = \sum_B P( A, B) \end{align} or, for three variables \begin{align} P(A, B) = \sum_C P(A, B, C) \end{align} You can think of the sum rule as either introducing another variable (from the left side of the equal sign to the right side) or removing a variable (from the right side of the equal sign to the left side). For this reason, we sometimes talk about the sum rule as "summing out" a variable (or "integrating out" for continuous variables). In the above equation, we're "summing out" $C$.

The forward algorithm relies on the sum rule to sum out the previous state: \begin{align} \alpha_k(z_k) & = P(z_k, x_{1:k}) \\ & = \sum_{z_{k-1}} P( z_k, x_{1:k}, z_{k-1}) \end{align} and the backward algorithm relies on the sum rule to sum out the next state: \begin{align} \beta_k(z_k) & = P(x_{k+1:n} | z_k) \\ & = \sum_{z_{k+1}} P(x_{k+1:n}, z_{k+1} | z_k) \end{align} Notice that both algorithms use the sum rule in exactly the same way; they just choose a different variable to sum out. Notice also that the backward algorithm does not introduce conditioning information; we're conditioning on $z_k$ in both lines.

$\endgroup$
2
  • $\begingroup$ Thanks for the rule. Can I verify if these two rules are correct? 1. P(A,B,C) = P(A|B,C)P(B,C) and 2. P(A,B|C) = P(A|B,C)P(B|C) $\endgroup$
    – raaj
    Dec 1 '18 at 23:23
  • $\begingroup$ Yes, those are both correct. They are both instances of the definition of conditional probability: $P(A | B, C) = \frac{P(A, B, C)}{ P(B, C)}$, and $P( A | B, C ) = \frac{P(A, B | C ) }{ P( B | C) }$ $\endgroup$
    – jkpate
    Dec 1 '18 at 23:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.