Assume that we have a random vector $\mathbf{Y}=(Y_1,...,Y_n)$ with unknown joint distribution. We know however that the conditional distribution of an entry $Y_i$ given the rest of the vector, without the entry $Y_i$ ($Y_{-i}$) is normal: $$ Y_i|Y_{-i}\sim N(\mu+\beta Y_kY_j,\sigma^2) $$ with $k,j\neq i$ standing in some relation with $i$. For example $k=i-1, j=k+1$ and $\mu$ some constant. Is it possible to find the joint distribution function? I.e. if for example we would have $$ Y_i|Y_{-i}\sim N(\mu+\beta Y_k,\sigma^2) $$ then the joint probability function would be a multivariate normal (with some resitriction on the parameter space (i.e. $\beta<1$), but what about the case above? I do understand that the joint distribution cannot be normal but isn't it possible that the joint distribution is just another multivariate distribution?

A short Gibbs sampling code in R shows that it is at least plausible that such a distribution exists as the sampler does not diverge (depending on the parameter space):

for (i in 1:10000){


To determine whether or not conditional distributions are compatible with one another, an approach is to apply the Hammersley-Clifford Theorem:

Under the condition that the supports of the full conditionals $g_{j}$ are the entire sets $\mathcal{Y}_j$, the joint distribution $g$ satisfies $$ g(y_1,\ldots,y_p) \propto \prod_{j=1}^p \; {g_{\ell_j}(y_{\ell_j}|y_{\ell_1}, \ldots,y_{\ell_{j-1}},y_{\ell_{j+1}}^\prime,\ldots,y_{\ell_p}^\prime) \over g_{\ell_j}(y_{\ell_j}^\prime|y_{\ell_1},\ldots,y_{\ell_{j-1}}, y_{\ell_{j+1}}^\prime,\ldots,y_{\ell_p}^\prime)} $$ for every permutation $\ell$ on $\{1,2,\ldots,p\}$ and every $y,y' \in \mathcal{Y}=\prod_{j=1}^p \mathcal{Y}_j$.

In the current setting, it however is impossible to find a joint distribution: the products $y_i y_j y_k$ appearing in the joint mean that the joint cannot be Normal. But since all conditionals are Normal, there is no other joint distribution possible.

Take as an example \begin{align*}X|Y,Z&\sim\mathcal{N}(\beta YZ,1)\\ Y|X,Z&\sim\mathcal{N}(\beta XZ,1)\\ Z|Y,X&\sim\mathcal{N}(\beta XY,1) \end{align*} and apply Hammersley-Clifford's formula with $x'=y'=z'=0$. The joint does not depend on the choice of $(x',y',z')$ and this choice makes a lot of terms vanish. The log joint density should be (up to an additive constant) $$-\{x^2+y^2+(z-\beta xy)^2-\beta^2 x^2 y^2\}/2$$ since all other terms cancel. Which means that the joint density is proportional to $$\exp(-\{x^2+y^2+(z-\beta xy)^2-\beta^2 x^2 y^2\}/2)$$ If I integrate this density in $z$, the term $(z-\beta xy)^2$ vanishes and integrate to a constant in $(x,y)$. I am thus left with $$\exp(-\{x^2+y^2-\beta^2 x^2 y^2\}/2)$$ which does not integrate in $x$ conditional on $y$ when $y^2>\beta^{-2}$. Therefore there cannot exist a joint probability density.

  • $\begingroup$ I think I am understanding your comment as well as the hint on the HC-Theorem. But the most important point for me is the last sentence. Is it true that normal conditionals must imply a joint normal? Clearly with the setting above the conditionals cannot imply a joint normal but why is it impossible that they are forming another joint distribution? $\endgroup$ – Michael L. Nov 16 '17 at 15:00
  • $\begingroup$ As I am pretty sure that gaussian conditionals can end up to form a non-gaussian joint distribution. I have updated my question. $\endgroup$ – Michael L. Nov 17 '17 at 8:04
  • $\begingroup$ Start your Gibbs sampler at $X=Y=Z=9$ and then you will watch it diverge very quickly! $\endgroup$ – Xi'an Nov 17 '17 at 8:13
  • $\begingroup$ Yes thats correct but only depending on the parameter space. I.e. with beta=-0.1 it will not diverge. Again I do not say that for all combinations of parameters a joint distribution exists but only for some cases. Furthermore I am still not convinced that gaussian conditionals must end up in a joint normal. $\endgroup$ – Michael L. Nov 17 '17 at 8:33
  • $\begingroup$ With other starting values, Gibbs also diverges for $\beta=-1$. See my update explaining why there cannot be a joint. $\endgroup$ – Xi'an Nov 17 '17 at 8:56

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.