Block sampling hidden state using forward algorithm only

Question

In a hidden Markov model, I can't get my mind around why I can't sample the full hidden state $\vec x$ using only a forward sampling algorithm.

Let $\vec y$ be the observed data and $\theta$ the model parameters. Let each time step $t$ be an independent realization so that $p(\vec y|\vec x, \theta) = \prod_t p(y_t|x_t, \theta)$. Also assume that $p(\vec x|\theta) = p(\vec x) = \prod_tp(x_t|x_{t-1})$, since I assume $x_t$ only depends on $x_{t-1}$.

I can write $p(\vec x|\theta, \vec y) = p(\vec y|\theta, \vec x) p(\theta, \vec x)/p(\vec y, \theta) \propto p(\vec y|\theta, \vec x)p(\vec x) = \prod_tp(y_t|x_t, \theta)p(x_t|x_{t-1})$.

Doesn't this mean I can simply sample using the forward algorithm, since I can rewrite $p(\vec x|\theta, \vec y) = \prod_tp(x_t|x_{t-1},y_t,\theta)$, where $p(x_t|x_{t-1},y_t,\theta) \propto p(y_t|x_t, \theta)p(x_t|x_{t-1})$? Since the label of states is arbitrary I can just set $x_0=0$ and run the sampler forward.

I'm sure something is wrong with my math, because I'm just sampling each $x_t$ using past information, which shouldn't give me a marginal sample of $\vec x$, but I can't pinpoint where my error is.

Answer 1

So here's what I missed. I cannot rewrite $p(x_t|x_{t-1}, y_t, \theta) \propto p(y_t|x_t, \theta)p(x_t|x_{t-1})$, since the proportionality constant is actually a function of $x_{t-1}$. So sampling from this distribution would give me a vector with distribution $p(\vec{x}|\vec{y}, \theta) = \prod_t p(y_t|x_t, \theta)p(x_t,x_{t-1})f(y_t,x_{t-1})$ which is different from my target distribution.

In particular $f(y_t,x_{t-1}) = \frac{1}{\sum_ip(y_t|i,\theta)p(i|x_{t-1})}$. I guess I should be more careful with this factorization technique and with omitting constants that turn out to be not so constant.