1
$\begingroup$

Very close to: Finding the Bayesian classifier for a bivariate Gaussian distribution

Can somebody tell me the joint Gaussian of the following two Gaussians (in which $C_j = C_1 \text{ or } C_2$):

$$p(x|C_1) = \text{N}(μ_1,Σ_1)$$

$$p(x|C_2) = \text{N}(μ_2,Σ_2)$$

where

$$μ_1 = \left[\begin{array}{c} 1\\ 1\end{array}\right], \,\,\,\, Σ_1 = \left[\begin{array}{c} \begin{array}{cc} 2 & 0\\ 0 & 1\end{array}\end{array}\right]$$

$$μ_2 = \left[\begin{array}{c} 1\\ 3\end{array}\right],\,\,\,\,Σ_2 = \left[\begin{array}{c} \begin{array}{cc} 2 & 0\\ 0 & 1\end{array}\end{array}\right]$$

$\endgroup$
  • 2
    $\begingroup$ could you change the title, please? "Joint Gaussian" is not what you mean. And if this is an homework, please tag it as such. $\endgroup$ – Xi'an Mar 18 '12 at 9:12
4
$\begingroup$

Your comments above lead me to believe that you are asking about the distribution of $X$ marginalized over the classes. This marginal distribution is Gaussian in the first dimension but not in the second.

Since the first and second dimensions are independent they can be treated separately. The means and variances conditional on class in the first dimension are the same for both classes, so mean and variance marginalized over class are the same those conditional on class $(\mu_1 = 1,\,\sigma_{1}^2 = 2)$. In the second dimension the marginal distribution is a mixture of Gaussians, which is not itself Gaussian:

$p(x_2) = \frac{1}{2}\text{N}(1,1) + \frac{1}{2}\text{N}(3,1),$

in which $\text{N}(\mu, \sigma^2)$ is the probability density of the Gaussian (aka normal) distribution with mean $\mu$ and variance $\sigma^2$.

The mean of the second dimension variable is

$\text{E}(X_2) = \Pr(C_1)\text{E}(X_2|C_1) + \Pr(C_2) \text{E}(X_2|C_2)$

$\text{E}(X_2) = \frac{1}{2} \cdot 1 + \frac{1}{2} \cdot 3 = 2.$

And now the variance. In general,

$\text{Var}(Y) = \text{E}(Y^2) - \left[\text{E}(Y)\right]^2.$

We'll find it useful to rearrange this as

$\text{E}(Y^2) = \text{Var}(Y) + \left[\text{E}(Y)\right]^2.$

So

$\text{E}(X_2^2) = \Pr(C_1)\text{E}(X_2^2|C_1) + \Pr(C_2) \text{E}(X_2^2|C_2)$

$\text{E}(X_2^2) = \frac{1}{2}(\text{Var}(X_2|C_1) + \left[\text{E}(X_2|C_1)\right]^2$ $+\text{Var}(X_2|C_2) + \left[\text{E}(X_2|C_2)\right]^2)$

$\text{E}(X_2^2) = \frac{1}{2}(1 + 1^2 + 1 + 3^2) = 6.$

Now we can get

$\text{Var}(X_2) = \text{E}(X_2^2) - \left[\text{E}(X_2)\right]^2 = 6 - 2^2 = 2.$

Here's what the distribution of $X_2$ looks like, along with a Gaussian distribution with the same mean and variance.

enter image description here

$\endgroup$
  • $\begingroup$ Thank you very much for your extremely descriptive answer. $\endgroup$ – picmate Mar 18 '12 at 3:17
  • $\begingroup$ To signal that I've answered your question, click the checkmark next to my answer. $\endgroup$ – Cyan Mar 18 '12 at 3:32
3
$\begingroup$

You need to tell us more about the question.

You seem to have some two-dimension random variable $X=(X_1,X_2)$ which has conditional bivariate normal distributions where $X_1$ and $X_2$ are independent.

Assuming that $C_1$ occurs with some probability $q$ and $C_2$ with probability $1-q$ then $X_1$ has a Gaussian distribution with mean $1$ and variance $2$ while independently $X_2$ has a non-Gaussian distribution with mean $3-2q$ and variance $1+4q - 4q^2$.

$\endgroup$
  • $\begingroup$ Hi thanks for the prompt reply. The two classes C1 and C2 are equi-probable. $\endgroup$ – picmate Mar 18 '12 at 0:24
  • $\begingroup$ Additionally, is it correct to add the two matrices and arrive at the joint gaussian as: (μ1+μ2), (Σ1+Σ2) ? $\endgroup$ – picmate Mar 18 '12 at 0:38
  • 2
    $\begingroup$ Isn't this almost a duplicate of stats.stackexchange.com/questions/24772/… ? $\endgroup$ – Zen Mar 18 '12 at 4:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.