I was reading about HMM in C.M. Bishop's book Pattern Recognition and Machine Learning. I was going through the forward and backward algorithm using $\alpha$ & $\beta$

For forward messaging passing

$\alpha(z_n) = p(x_n|z_n)*\sum_{z_n-1}\alpha(z_{n-1})*P(z_n|z_{n-1})$

Since the probabilities are <=1. As we move from $z_1$ to $z_N$ the $\alpha(z_n)$ tends to get smaller and smaller and we might have floating point issues. For that the book mentioned something called scaling factors where u normalize the $\alpha$

So we have

$\hat{\alpha(z_n)} = p(z_n|x_1...x_n) = \alpha(z_n)/p(x_1...x_n)$

which we expect to be well behaved numerically because it is a probability distribution over K variables for any value of n. I didn't get how it is well behaved and how it solves the floating point issue and what does it mean by probability distribution over K variables. I just didn't get this part


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.