I am approximating a probability distribution over n RVs by n factors following a bivariate conditional distribution. For instance for 4 variables I could factorize p as:

$$p(x_1,x_2,x_3,x_4) = p(x_1)p(x_2|x_1)p(x_3|x_1)p(x_4|x_2)$$

The question is how to obtain for instance the marginal $p(x_1,x_3,x_4)$?

$p(x_1)$ is a normal distribution, while all other conditional are conditional bivariate normal distributions.

  • $\begingroup$ I would like to marginalize over $x_2$, and I want to write the new factorization $\endgroup$
    – alej
    Jun 14 '18 at 15:48
  • 1
    $\begingroup$ Integrating $x_2$ out provides the marginal (Normal) multivariate density. $\endgroup$
    – Xi'an
    Jun 14 '18 at 16:04
  • $\begingroup$ Your factorization leaves much to be desired since it involves a multitude of unstated (and possibly contradictory) assumptions. That "I could factorize" hides a lot. Why can you factorize as you say you can? It is not a standard result my any means. Also, what do you know exactly? Do you know the actual distributions that you call $p(x_2\mid x_1)$, $p(x_3\mid x_1)$, $p(x_4\mid x_2)$ or just generically as in "the conditional distribution of $x_3$ given $x_1$ is a normal distribution", i.e. without specification of mean and/or variance? $\endgroup$ Jun 14 '18 at 18:30

In the event $$p(x_1,x_2,x_3,x_4) = p(x_1)p(x_2|x_1)p(x_3|x_1)p(x_4|x_2)$$ only involve Normal distributions, one can without loss of generality assume that all four means are zero and all four variances are one, since otherwise, we can return to this case by a change of variables. If (and this is unclear from the question as stated) the three bivariate distributions defined by \begin{align*} p(x_1,x_2)&=p(x_1)p(x_2|x_1)\\ p(x_1,x_3)&=p(x_1)p(x_3|x_1)\\ p(x_2,x_4)&=p(x_2)p(x_4|x_2)\\ \end{align*} are bivariate Normal with correlation coefficients $\rho_{12}$, $\rho_{13}$, $\rho_{24}$, then \begin{align*} \log p(x_1,x_2)&\propto -\frac{1}{2}x_1^2-\frac{1}{2}x_2^2+\rho_{12}x_1x_2\\ \log p(x_1,x_3)&\propto-\frac{1}{2}x_1^2-\frac{1}{2}x_3^2+\rho_{13}x_1x_3\\ \log p(x_2,x_4)&\propto-\frac{1}{2}x_2^2-\frac{1}{2}x_4^2+\rho_{24}x_2x_4\\ \end{align*} (where the proportionality factor unusually means up to an additive constant). Now $$p(x_2|x_1)p(x_4|x_2)\propto \exp\left\{- \frac{x_2^2}{2(1-\rho_{12}^2)}+\frac{\rho_{12}x_1x_2}{(1-\rho_{12}^2)}-\frac{x_4^2}{2(1-\rho_{24}^2)}+\frac{\rho_{24}x_2x_4}{(1-\rho_{24}^2)} \right\}$$ (where the proportionality factor is back to its usual meaning up to a multiplicative constant). Hence \begin{align*}p(x_4|x_2) &\propto \int_{-\infty}^{\infty} \exp\left\{- \frac{x_2^2}{2(1-\rho_{12}^2)}+\frac{\rho_{12}x_1x_2}{(1-\rho_{12}^2)}-\frac{x_4^2}{2(1-\rho_{24}^2)}+\frac{\rho_{24}x_2x_4}{(1-\rho_{24}^2)} \right\}\text{d}x_2\\ &\propto \exp\left\{-\frac{x_4^2}{2(1-\rho_{24}^2)}\right\} \\ &\quad \times \int_{-\infty}^{\infty} \exp\left\{- \frac{x_2^2}{2(1-\rho_{12}^2)}+\frac{\rho_{12}x_1x_2}{(1-\rho_{12}^2)}+\frac{\rho_{24}x_2x_4}{(1-\rho_{24}^2)} \right\}\text{d}x_2\\ &\propto \exp\left\{-\frac{x_4^2}{2(1-\rho_{24}^2)}\right\} \\ &\quad \times \int_{-\infty}^{\infty} \exp\frac{-1}{2(1-\rho_{12}^2)}\left\{x_2^2-2\rho_{12}x_1x_2-\frac{2(1-\rho_{12}^2)\rho_{24}x_2x_4}{(1-\rho_{24}^2)} \right\}\text{d}x_2\\ &\propto \exp\left\{-\frac{x_4^2}{2(1-\rho_{24}^2)}\right\} \\ &\quad \times \int_{-\infty}^{\infty} \exp\frac{-1}{2(1-\rho_{12}^2)}\left\{\left(x_2-\rho_{12}x_1-\frac{(1-\rho_{12}^2)\rho_{24}x_4}{(1-\rho_{24}^2)} \right)^2\right\}\text{d}x_2\\ &\quad\times \exp\frac{+ 1}{2(1-\rho_{12}^2)}\left\{\left( \rho_{12}x_1 + \frac{(1-\rho_{12}^2)\rho_{24}x_4}{(1-\rho_{24}^2)} \right)^2\right\}\\ &\propto \exp\left\{-\frac{x_4^2}{2(1-\rho_{24}^2)}\right\} \\ &\quad \times \int_{-\infty}^{\infty} \exp\frac{-1}{2(1-\rho_{12}^2)}\left\{\left(x_2-\rho_{12}x_1-\frac{(1-\rho_{12}^2)\rho_{24}x_4}{(1-\rho_{24}^2)} \right)^2\right\}\text{d}x_2\\ &\quad \times \exp\frac{+ 1}{2(1-\rho_{12}^2)}\left\{\left( \rho_{12}x_1 + \frac{(1-\rho_{12}^2)\rho_{24}x_4}{(1-\rho_{24}^2)} \right)^2\right\}\\ &\propto \exp\left\{-\frac{x_4^2}{2(1-\rho_{24}^2)}\right\} \\ &\quad \times \exp\frac{+ 1}{2(1-\rho_{12}^2)}\left\{\left( \rho_{12}x_1 + \frac{(1-\rho_{12}^2)\rho_{24}x_4}{(1-\rho_{24}^2)} \right)^2\right\} \end{align*}

  • $\begingroup$ I don't understand how this gives the desired $p(x1,x_3,x_4)$. $\endgroup$ Jun 14 '18 at 18:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.