HMM - Difference between forward and backward case

In the HMM formulation where z is hidden state and x is observed

In the forward case, I see it represented as such:

$$\alpha_{k}(z_{k})=P(z_{k},x_{1:k})=\sum_{z_{k-1}}P(z_{k},z_{k-1},x_{1:k})$$

but in the backward case I see it as such

$$\beta_{k}(z_{k})=P(x_{k+1:n}|z_{k})=\sum_{z_{k+1}}P(x_{k+1:n},z_{k+1}|z_{k})$$

I thought that in the probability rules, we can factor in a new term based on this expression:

$$P(A,B)=\sum_{C}P(A,C|B)$$

If so, how are both of those expressions even expanded? Shoudn't the forward rule then be:

$$\alpha_{k}(z_{k})=P(z_{k},x_{1:k})=\sum_{z_{k-1}}P(z_{k},z_{k-1}|x_{1:k})$$

and the backward rule be not possible to factorize based on the above rule? Is my rule wrong?

• The rule is not correct. Could you provide more information on why you think it is a valid rule so I can provide a better answer? Dec 1 '18 at 22:42
• Then can I know the correct rule please? I need to know the general rule that allows for the formulation of those two backward forward expressions.
– raaj
Dec 1 '18 at 22:48

The probability rule is not right. I think you are thinking of the sum rule: \begin{align} P( A ) = \sum_B P( A, B) \end{align} or, for three variables \begin{align} P(A, B) = \sum_C P(A, B, C) \end{align} You can think of the sum rule as either introducing another variable (from the left side of the equal sign to the right side) or removing a variable (from the right side of the equal sign to the left side). For this reason, we sometimes talk about the sum rule as "summing out" a variable (or "integrating out" for continuous variables). In the above equation, we're "summing out" $$C$$.
The forward algorithm relies on the sum rule to sum out the previous state: \begin{align} \alpha_k(z_k) & = P(z_k, x_{1:k}) \\ & = \sum_{z_{k-1}} P( z_k, x_{1:k}, z_{k-1}) \end{align} and the backward algorithm relies on the sum rule to sum out the next state: \begin{align} \beta_k(z_k) & = P(x_{k+1:n} | z_k) \\ & = \sum_{z_{k+1}} P(x_{k+1:n}, z_{k+1} | z_k) \end{align} Notice that both algorithms use the sum rule in exactly the same way; they just choose a different variable to sum out. Notice also that the backward algorithm does not introduce conditioning information; we're conditioning on $$z_k$$ in both lines.
• Yes, those are both correct. They are both instances of the definition of conditional probability: $P(A | B, C) = \frac{P(A, B, C)}{ P(B, C)}$, and $P( A | B, C ) = \frac{P(A, B | C ) }{ P( B | C) }$ Dec 1 '18 at 23:26