When applying Bayes' Theorem to observed variable $\mathbf{x} = (x_1, x_2, \ldots, x_n)$ and latent variable $\mathbf{z} = (z_1, z_2,\ldots, z_m)$, assuming a prior $p(\mathbf{z}) = \prod_{i=1}^m p(z_i)$ which factorizes over dimensions, does this imply the posterior $p(\mathbf{z} \vert \mathbf{x})$ is also factorized?

$$ p(\mathbf{z} \vert \mathbf{x}) = \frac{p(\mathbf{x} \vert \mathbf{z}) p(\mathbf{z})}{p(\mathbf{x})} = \frac{p(\mathbf{x} \vert \mathbf{z})}{p(\mathbf{x})} \prod_{i=1}^m p(z_i)$$

If this is the case, how does the probability mass/density of the factor $\frac{p(\mathbf{x} \vert \mathbf{z})}{p(\mathbf{x})}$ 'distribute' itself over the dimensions of $p(\mathbf{z}) = \prod_{i=1}^m p(z_i)$ - equally amongst all dimensions?

In other words, if the posterior can be written as:

$$ p(\mathbf{z} \vert \mathbf{x}) = \prod_{i=1}^m p(z_i \vert \mathbf{x}) =\prod_{i=1}^m \alpha_i p(z_i)$$

What would the $\alpha_i$ be?

  • $\begingroup$ By factorisation of $p(z|x)$, I'd assume something like $\prod p(z_i|x)$,but the first equation you wrote is always correct given that $p(z)$ is factorized. $\endgroup$ – gunes Oct 12 at 13:22
  • $\begingroup$ Thanks, I edited the question to make it more clear, but I'm not sure if that directly translates to factorization of the posterior. $\endgroup$ – Eweler Oct 12 at 13:32

There is no reason for $p(\mathbf x|\mathbf z)$ to factorise in a product of functions of the $z_i$ (for a given $\mathbf x$). The OP seems to ignore the fact that $p(\mathbf x|\mathbf z)$ is a function of $\mathbf z$ as a whole and hence that$$p(\mathbf x|\mathbf z)\prod_{i=1}^m p(z_i)$$has no reason to turn into$$\prod_{i=1}^m \alpha_i p_i(z_i)$$unless the $\alpha_i$'s are themselves functions of $\mathbf z$, such that $$\prod_{i=1}^m \alpha_i(\mathbf z) \propto p(\mathbf x|\mathbf z)$$ which ruins the purpose of the decomposition.

| cite | improve this answer | |

Not the answer you're looking for? Browse other questions tagged or ask your own question.