4
$\begingroup$

Suppose I have a set of variables $(x_1, x_2, x_3)$ and I know that all the conditionals are Gaussian. That is, I know that $p(x_1)$, $p(x_2\mid x_1)$ and $p(x_3 \mid x_2, x_1)$ are Gaussian.

What can we say about the joint distribution $p(x_1, x_2, x_3)$? It is necessarily Gaussian?

Context

I am reading Normalizing Flows for Probabilistic Modeling and Inference. If you go to 3.1.1., in the paragraph below equation (33) we have the following set up:

Suppose I have $\boldsymbol{z}=(z_1, z_2, z_3)$ jointly Gaussian, and I transform this into a random variable $\boldsymbol{z}' = (z_1', z_2', z_3')$ as follows:

  • $z_1 ' = \alpha_1z_1 + \beta_1$ where $\alpha_1$ and $\beta_1$ are fixed.
  • $z_2' = \alpha_2 z_2 + \beta_2$ where $\{\alpha_2, \beta_2\}$ are output of a neural network with input $\boldsymbol{z}_{< 2} = (z_1)$.
  • $z_3' = \alpha_3z_3 + \beta_3$ where $\{\alpha_3, \beta_3\}$ are the output of a neural network with input $\boldsymbol{z}_{<3} = (z_1, z_2)$

Then we know:

  • $z_1$ is Gaussian since it's a marginal of a Gaussian.
  • $z_1'$ is Gaussian since it's an affine transformation of a Gaussian.

Then, they say, this means that all the conditionals $z_i'\mid \boldsymbol{z}_{< i}'$ are Gaussian however the joint $\boldsymbol{z}'$ is not.

$\endgroup$
  • 3
    $\begingroup$ Consider the simpler question with just two variables. For instance, suppose the distribution of $x_2$ given $x_1$ is standard Gaussian when $x_1\lt 0$ and otherwise is Gaussian with mean $10$ and unit variance: is $(x_1,x_2)$ jointly Gaussian? $\endgroup$ – whuber Jul 28 at 16:14
  • 2
    $\begingroup$ Example in Q does not have normal $X_2$ marginal. $\endgroup$ – BruceET Jul 28 at 16:30
  • 2
    $\begingroup$ The problem with your situation is that $p(x_1 \vert x_2)$ can mean many things. You have a counter example now. But maybe you were more specifically thinking about a different conditional distribution? E.g. when $p(x_1 \vert x_2) \sim N(a+b x_2, \sigma^2)$ then the joint distribution is a multivariate Gaussian. $\endgroup$ – Sextus Empiricus Jul 28 at 16:33
  • 1
    $\begingroup$ @BruceET $X_2$ is not required to be normal marginal, it's only required to be normal conditional on $X_1$, which it is in whuber's example $\endgroup$ – Jake Westfall Jul 28 at 16:38
  • 2
    $\begingroup$ @SextusEmpiricus If you're referring to my comment to BruceET, my point is that his objection does not invalidate whuber's counterexample to the general question as written. The fact that whuber's example does not lead to all normal marginals is not a problem because the question does not suppose that all the marginals are normal. In fact, it's simply another way of recognizing the validity of the counterexample: it satisfies the premises of the question (as written above Context), but it doesn't even lead to normal marginals, and so clearly doesn't lead to a multivariate normal $\endgroup$ – Jake Westfall Jul 28 at 17:53
3
$\begingroup$

This answer is partial, and based on Whuber's comment: Consider the simpler question with just two variables. For instance, suppose the distribution of $x_2$ given $x_1$ is standard Gaussian when $x_1 < 0$ and otherwise is Gaussian with mean 10 and unit variance: is $(x_1, x_2)$ jointly Gaussian?

This is how I generated the two random variables.

x1 <- rnorm(n=1000, mean=0, sd=1)
x2 <- rep(0, 1000)
for (i in 1:1000){
  if (x1[i] < 0){
    x2[i] = rnorm(n=1)
  } else {
    x2[i] = rnorm(n=1, mean=10)
  }
}

And this is how I plot them

library(ggplot2)
df <- data.frame(x1=x1, x2=x2)
ggplot(data=df) + 
  geom_point(aes(x=x1, y=x2))

Obtaining the following figure:

enter image description here

which looks quite different from a bivariate Normal distribution.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ This reflects a joint distribution--but it's obviously not bivariate Normal. (Using simulation and graphics in this way is very effective and obviates any need for a mathematical demonstration, I hope!) $\endgroup$ – whuber Jul 28 at 18:19
  • $\begingroup$ @Westfall I’ve just included it! $\endgroup$ – Euler_Salter Jul 29 at 7:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.