# Combination of two Gaussians

Can somebody tell me the joint Gaussian of the following two Gaussians (in which $C_j = C_1 \text{ or } C_2$):

$$p(x|C_1) = \text{N}(μ_1,Σ_1)$$

$$p(x|C_2) = \text{N}(μ_2,Σ_2)$$

where

$$μ_1 = \left[\begin{array}{c} 1\\ 1\end{array}\right], \,\,\,\, Σ_1 = \left[\begin{array}{c} \begin{array}{cc} 2 & 0\\ 0 & 1\end{array}\end{array}\right]$$

$$μ_2 = \left[\begin{array}{c} 1\\ 3\end{array}\right],\,\,\,\,Σ_2 = \left[\begin{array}{c} \begin{array}{cc} 2 & 0\\ 0 & 1\end{array}\end{array}\right]$$

-
could you change the title, please? "Joint Gaussian" is not what you mean. And if this is an homework, please tag it as such. –  Xi'an Mar 18 '12 at 9:12

Your comments above lead me to believe that you are asking about the distribution of $X$ marginalized over the classes. This marginal distribution is Gaussian in the first dimension but not in the second.

Since the first and second dimensions are independent they can be treated separately. The means and variances conditional on class in the first dimension are the same for both classes, so mean and variance marginalized over class are the same those conditional on class $(\mu_1 = 1,\,\sigma_{1}^2 = 2)$. In the second dimension the marginal distribution is a mixture of Gaussians, which is not itself Gaussian:

$p(x_2) = \frac{1}{2}\text{N}(1,1) + \frac{1}{2}\text{N}(3,1),$

in which $\text{N}(\mu, \sigma^2)$ is the probability density of the Gaussian (aka normal) distribution with mean $\mu$ and variance $\sigma^2$.

The mean of the second dimension variable is

$\text{E}(X_2) = \Pr(C_1)\text{E}(X_2|C_1) + \Pr(C_2) \text{E}(X_2|C_2)$

$\text{E}(X_2) = \frac{1}{2} \cdot 1 + \frac{1}{2} \cdot 3 = 2.$

And now the variance. In general,

$\text{Var}(Y) = \text{E}(Y^2) - \left[\text{E}(Y)\right]^2.$

We'll find it useful to rearrange this as

$\text{E}(Y^2) = \text{Var}(Y) + \left[\text{E}(Y)\right]^2.$

So

$\text{E}(X_2^2) = \Pr(C_1)\text{E}(X_2^2|C_1) + \Pr(C_2) \text{E}(X_2^2|C_2)$

$\text{E}(X_2^2) = \frac{1}{2}(\text{Var}(X_2|C_1) + \left[\text{E}(X_2|C_1)\right]^2$ $+\text{Var}(X_2|C_2) + \left[\text{E}(X_2|C_2)\right]^2)$

$\text{E}(X_2^2) = \frac{1}{2}(1 + 1^2 + 1 + 3^2) = 6.$

Now we can get

$\text{Var}(X_2) = \text{E}(X_2^2) - \left[\text{E}(X_2)\right]^2 = 6 - 2^2 = 2.$

Here's what the distribution of $X_2$ looks like, along with a Gaussian distribution with the same mean and variance.

-
Thank you very much for your extremely descriptive answer. –  picmate Mar 18 '12 at 3:17
To signal that I've answered your question, click the checkmark next to my answer. –  Cyan Mar 18 '12 at 3:32
You seem to have some two-dimension random variable $X=(X_1,X_2)$ which has conditional bivariate normal distributions where $X_1$ and $X_2$ are independent.
Assuming that $C_1$ occurs with some probability $q$ and $C_2$ with probability $1-q$ then $X_1$ has a Gaussian distribution with mean $1$ and variance $2$ while independently $X_2$ has a non-Gaussian distribution with mean $3-2q$ and variance $1+4q - 4q^2$.