3
$\begingroup$

I am new to machine learning and have been reading about gibbs sampling. From my understanding, a Gibbs algorithm samples a single variable iteratively conditioned on all other variables. In blocked Gibbs sampling, I block a subset of variables and sample iteratively conditioned on other variables. I understand both Gibbs and blocked Gibbs but I am confused on why Forward-Backward algorithm is needed for blocked Gibbs on a HMM. Can somone clarify my confusion? Could someone also provide example of forward/backward as it relates to Gibbs sampling?

$\endgroup$
1
$\begingroup$

Let's call your parameters $\theta$, your time series of observations $y_{1:T}$, and your hidden states $x_{1:T}$.

Blocked-Gibbs sampling targets the full posterior $p(\theta,x_{1:T}|y_{1:T})$. It alternates between sampling $X_{1:T}^i \sim p(x_{1:T}|y_{1:t},\theta)$ and $\theta^i \sim p(\theta|x_{1:T},y_{1:T})$. It is called Blocked Gibbs because all of the states are blocked together and sampled all at once. You need the Forward-Backward algorithm to do the first one: to sample from $p(x_{1:T}|y_{1:t},\theta)$.

And here's a paper.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.