# Observations for a bivariate Gaussian mixture

Consider two random vectors $X\equiv(X_1, X_2),Y\equiv(Y_1, Y_2)$ distributed as below

1) $X\sim N(\begin{pmatrix} \mu_{X,1}\\ \mu_{X,2}\\ \end{pmatrix}, \begin{pmatrix} v_{X,1} & 0\\ 0 & v_{X,2} \end{pmatrix})$

2) $Y\sim N(\begin{pmatrix} \mu_{Y,1}\\ \mu_{Y,2}\\ \end{pmatrix}, \begin{pmatrix} v_{Y,1} & 0\\ 0 & v_{Y,2} \end{pmatrix})$

Consider now the random vector $W\equiv(W_1, W_2)$ whose probability distribution is obtained by mixing $X,Y$ with equal weights $1/2$, i.e. $$f_W=\frac{1}{2}f_X+ \frac{1}{2}f_Y$$ where $f$ denotes the pdf.

Questions: under which sufficient conditions $W_1\perp W_2$, where $\perp$ denotes independence?

MY ATTEMPTS: Working on this problem I ended up finding some necessary and sufficient conditions for independence between $W_1$ and $W_2$. But I'm suspicious and I am asking you help to find a mistake.

Let $M_{W_1, W_2}$ denote the moment generating function of $W_1, W_2$. By construction $$M_{W_1, W_2}(s,t)=\frac{1}{2}\exp(s\mu_{X,1}+t\mu_{X,2}+s^2v_{X,1}+t^2v_{X,2})$$ $$+\frac{1}{2}\exp(s\mu_{Y,1}+t\mu_{Y,2}+s^2v_{Y,1}+t^2v_{Y,2})$$

Proposition: $M_{W_1, W_2}(s,t)=M_{W_1, W_2}(s,0)\times M_{W_1, W_2}(0,t)$ $\forall (s,t)\in \mathbb{R}^2$ if and only if $W_1$ independent of $W_2$

In my case $$M_{W_1, W_2}(s,0)\times M_{W_1, W_2}(0,t)=\frac{1}{4}\exp(s\mu_{X,1}+t\mu_{X,2}+s^2v_{X,1}+t^2v_{X,2})$$ $$+\frac{1}{4}\exp(s\mu_{Y,1}+t\mu_{Y,2}+s^2v_{Y,1}+t^2v_{Y,2})$$ $$+\frac{1}{4}\exp(s\mu_{X,1}+t\mu_{Y,2}+s^2v_{X,1}+t^2v_{Y,2})$$ $$+\frac{1}{4}\exp(s\mu_{Y,1}+t\mu_{X,2}+s^2v_{Y,1}+t^2v_{X,2})$$ Notice that $$M_{W_1, W_2}(s,0)\times M_{W_1, W_2}(0,t)=M_{W_1, W_2}(s,t)$$ if and only if

$\mu_{X,1}=\mu_{Y,1}$ and $v_{X,1}=v_{Y,1}$

or

$\mu_{X,2}=\mu_{Y,2}$ and $v_{X,2}=v_{Y,2}$

Therefore, by the proposition above, $W_1 \perp W_2$ if and only if

$\mu_{X,1}=\mu_{Y,1}$ and $v_{X,1}=v_{Y,1}$

or

$\mu_{X,2}=\mu_{Y,2}$ and $v_{X,2}=v_{Y,2}$.

• Are $X$ and $Y$ independent vectors? – Alecos Papadopoulos Nov 5 '17 at 16:10
• I agree with your conclusion that the two components are independent iff the mixture is not a mixture wrt one of the two components. Otherwise, they are necessarily dependent. – Xi'an Nov 5 '17 at 16:13

In an alternative way you could use the requirement and sufficient condition that you have independence iff the combined probability of $X_1$ and $X_2$ is the product of the probability of $X_1$ and $X_2$.

$$f_{(X_1,X_2)} = f_{X_1} f_{X_2}$$

And this you could work out more easily (more easily in the sense that the part after your 'notice that...' is not so trivial):

$W$ as mixture of $X$ and $Y$ $$f_{(W_1,W_2)} = a f_{(X_1,X_2)} + b f_{(Y_1,Y_2)}$$

$W$ as product of PDFs of mixture of $X_1$ and $Y_1$ and mixture of $X_2$ and $Y_2$ (only iff $W_1$ and $W_2$ are independent) $$f_{(W_1,W_2)} = \hphantom{\frac{1}{2}} f_{W_1}f_{W_2} \hphantom{+ \frac{1}{2} f_{(Y_1,Y_2)}}$$

Which leads to: $$a f_{X_1}f_{X_2} + b f_{Y_1}f_{Y_2} = (a f_{X_1} + b f_{Y_1})(a f_{X_2}+ b f_{Y_2})$$

And, for any $a+b=1$, you get after some algebra:

$$(f_{X_1}-f_{Y_1}) (f_{X_2}-f_{Y_2}) = 0$$

So either $$f_{X_1} = f_{Y_1} \qquad or \qquad f_{X_2} = f_{Y_2}$$

Note that this generalizes your results to other multivariate distributions, allows adding more variables e.g. if $f_{X_1} = f_{Y_1} = f_{Z_1}$, and allows to use different mixing ratios.

The algebra in steps:

starting from:

$$a f_{X_1}f_{X_2} + b f_{Y_1}f_{Y_2} = (a f_{X_1} + b f_{Y_1})(a f_{X_2}+ b f_{Y_2})$$

bring the terms on the right hand side to the left and group by $a f_{X_1}$ and $b f_{Y_1}$

$$a f_{X_1}(f_{X_2} - (a f_{X_2}+ b f_{Y_2}) ) + b f_{Y_1}(f_{Y_2}- (a f_{X_2}+ b f_{Y_2})) = 0$$

simplify using $a+b=1$

$$a f_{X_1}(bf_{X_2} - (b f_{Y_2}) ) + b f_{Y_1}(a f_{Y_2}- (a f_{X_2})) = 0$$

divide by $a$ and $b$

$$f_{X_1}(f_{X_2} - f_{Y_2} ) + f_{Y_1}(f_{Y_2}- f_{X_2}) = 0$$