With complete data and a factored prior, the posterior also factors

Question

In the second paragraph of Section 11.3 in Machine Learning A Probabilistic Perspective, the author concisely summarizes Section 10.4.2 by saying that for the standard bayesian model $$P({\boldsymbol\theta}|D)=\frac{P(D|{\boldsymbol\theta})P({\boldsymbol\theta})}{P(D)}$$

...when we have complete data and a factored prior, the posterior over the parameters also factorizes...

Let's take a simple example. A single observation from a bivariate normal with known covariance matrix. The associated graphical model I have in mind is $$\mu_1\rightarrow x_1\rightarrow x_2\leftarrow \mu_2$$ where $\mu=(\mu_1,\mu_2)$ are the means and $x=(x_1,x_2)$ is the single observed data point. Note that this graphical model does in fact encode the desired independence: $(\mu_1\perp\mu_2)\;|\;(x_1,x_2)$.

For simplicities sake let's make the prior distribution uniform. Thus we have $$P(\mu)=N(0,\infty I)$$ $$P(x|\mu)=N(\mu, \Sigma)$$ where we choose the covariance matrix of the sample distribution to be $$\Sigma=\begin{bmatrix} 1 &\frac{1}{2} \\ \frac{1}{2} &1 \end{bmatrix}$$

Then the posterior distribution for $\mu$ is given by $$P(\mu|x)=N(x, \Sigma)$$ Here's the rub. The prior is clearly factorable, and the data is completely observed, yet the posterior is not factorable, since its covariance matrix is not diagonal.

So I must be incorrectly interpreting what the author means by the term 'complete data'. Could someone help me correct my misunderstanding here, it's been driving my nuts.

Answer 1

The notion of complete data is very weak, in that it simply means you observe the original model, rather than incomplete data. There is no conditional independence assumption behind this notion. In that sense, the quote from Section 11.3 is clearly confusing.

In equation (10.27) of Murphy'sMachine Learning, the sampling model is such that it factorises as $$\prod_t p(\mathcal{D}_t|\theta_t)$$ And Murphy rightly states that when the prior also factorises as $$\prod_t p(\theta_t)$$ the posterior reproduces this factorisation $$p(\theta|\mathcal{D})\propto \prod_t p(\mathcal{D}_t|\theta_t)p(\theta_t)$$ i.e. the $\theta_t$'s are independent a posteriori.

In your graphical model and Gaussian example, there is no such separation in the sampling model: $\mu_1$ and $\mu_2$ are independent a priori (if we ignore the issue with the infinite mass prior) but not a posteriori.