# Is it always possible to find a joint distribution $p(x_1,x_2,x_3,x_4)$ consistent with these local conditional distributions?

I am currently studying Bayesian Reasoning and Machine Learning by David Barber, the 4th chapter exercise 4.1 (p 79). The exercise is the following:

Exercise 4.1

1. Consider the pairwise Markov network, $$p(x) = \phi(x_1,x_2)\phi(x_2,x_3)\phi(x_3,x_4)\phi(x_4,x_1)$$ Express in terms of $$\phi$$ the following: $$p(x_1|x_2,x_4), p(x_2|x_1,x_3),p(x_3|x_2,x_4),p(x_4|x_1,x_3)$$
2. For a set of local distributions defined as $$p(x_1|x_2,x_4), p(x_2|x_1,x_3),p(x_3|x_2,x_4),p(x_4|x_1,x_3)$$ is it always possible to find a joint distribution $$p(x_1,x_2,x_3,x_4)$$ consistent with these local conditional distributions?

Now, I've done the whole first part. I have a solution manual and checked my derivations are correct. Here's the first expression:

$$p(x_1|x_2,x_4) = \frac{\sum_3\phi(x_1,x_2)\phi(x_2,x_3)\phi(x_3,x_4)\phi(x_4,x_1)}{\sum_{1,3}\phi(x_1,x_2)\phi(x_2,x_3)\phi(x_3,x_4)\phi(x_4,x_1)} \\ = \frac{\phi(x_1,x_2)\phi(x_4,x_1)\sum_3\phi(x_2,x_3)\phi(x_3,x_4)}{\sum_1\phi(x_1,x_2)\phi(x_4,x_1)\sum_3\phi(x_2,x_3)\phi(x_3,x_4)}\\ = \frac{\phi(x_1,x_2)\phi(x_4,x_1)}{\sum_1\phi(x_1,x_2)\phi(x_4,x_1)}$$

What I don't understand is the manual's answer to the second question, in particular how their equation is correct. I also don't understand the question itself.

It is not always possible to do this. One way to see this is to consider for example $$p_1(x_1|x_2,x_4)\sum_{x_1,x_3}p(x_1,x_2,x_3,x_4) = p(x_1,x_2,x_3,x_4)$$

What does $$p_1$$ mean?

How is this correct? Since as I've shown before

$$p(x_1|x_2,x_4) = \frac{\sum_3\phi(x_1,x_2)\phi(x_2,x_3)\phi(x_3,x_4)\phi(x_4,x_1)}{\sum_{1,3}\phi(x_1,x_2)\phi(x_2,x_3)\phi(x_3,x_4)\phi(x_4,x_1)} \\ \Leftrightarrow \\ p(x_1|x_2,x_4) = \frac{\sum_3 p(x_1,x_2,x_3,x_4)}{\sum_{1,3} p(x_1,x_2,x_3,x_4)} \\ \Leftrightarrow \\ p(x_1|x_2,x_4) \sum_{1,3} p(x_1,x_2,x_3,x_4)= \sum_3 p(x_1,x_2,x_3,x_4)$$

So how is that correct? Also, what does it mean for a distribution to be consistent exactly?

¡The reported exercise uses confusing notations in that $$p(\cdot)$$ denotes many different functions!
If one defines the conditional distributions$$p(x_1|x_2,x_4), p(x_2|x_1,x_3),p(x_3|x_2,x_4),p(x_4|x_1,x_3)$$arbitrarily, there is no reason these four (marginalised) conditional densities are compatible with a joint distribution. That is, there may exist no joint distribution such that these are its four conditionals.
On the other hand, the equation $$p(x_1|x_2,x_4)\sum_{x_1,x_3}p(x_1,x_2,x_3,x_4) = p(x_1,x_2,x_3,x_4)$$ is not correct, as the lhs integrates $$x_3$$ out, while the rhs exhibits an $$x_3$$. The correct equation is $$p(x_1|x_2,x_4)\underbrace{\sum_{x_1,x_3}p(x_1,x_2,x_3,x_4)}_\text{marginal of (x_2,x_4)} = \underbrace{p(x_1,x_2,x_4)}_\text{marginal of (x_1,x_2,x_4)}$$ assuming $$X_1$$ only depends on $$(X_2,X_4)$$, ie is conditionally independent of $$X_3$$.
According to the Hammersley-Clifford theorem, the joint (when it exists) is given by $$\dfrac{p(x_1,x_2,x_3,x_4)}{p(x^0_1,x^0_2,x^0_3,x^0_4)}= \dfrac{p(x_1|x_2,x_4)p(x_2|x_1^0,x_3)p(x_3|x_2^0,x_4)p(x_4|x_1^0,x_3^0)}{p(x_1^0|x_2,x_4)p(x_2^0|x_1^0,x_3)p(x_3^0|x_2^0,x_4)p(x^0_4|x^0_1,x^0_3)}$$ where $$(x_1^0,x^0_2,x_3^0,x^0_4)$$ is an arbitrary value (with positive probability). For the conditionals to be compatible, the rhs should factor as a function of $$(x_1^0,x^0_2,x_3^0,x^0_4)$$ times a function of $$(x_1,x_2,x_3,x_4)$$.
• See my editing. For instance, $p(x_1|x_2)$ and $p(x_2|x_1)$ are compatible with a joint distribution if and only if$$\dfrac{p(x_1|x_2)}{p(x_2|x_1)}=a(x_1)b(x_2)$$ie the ratio is factorising as the product of a function of $x_1$ and a function of $x_2$. Jul 10, 2022 at 15:15