6
$\begingroup$

Consider two random vectors $X\equiv(X_1, X_2),Y\equiv(Y_1, Y_2)$ distributed as below

1) $X\sim N(\begin{pmatrix} \mu_{X,1}\\ \mu_{X,2}\\ \end{pmatrix}, \begin{pmatrix} v_{X,1} & 0\\ 0 & v_{X,2} \end{pmatrix})$

2) $Y\sim N(\begin{pmatrix} \mu_{Y,1}\\ \mu_{Y,2}\\ \end{pmatrix}, \begin{pmatrix} v_{Y,1} & 0\\ 0 & v_{Y,2} \end{pmatrix})$

Consider now the random vector $W\equiv(W_1, W_2)$ whose probability distribution is obtained by mixing $X,Y$ with equal weights $1/2$, i.e. $$ f_W=\frac{1}{2}f_X+ \frac{1}{2}f_Y $$ where $f$ denotes the pdf.

Questions: under which sufficient conditions $W_1\perp W_2$, where $\perp$ denotes independence?


MY ATTEMPTS: Working on this problem I ended up finding some necessary and sufficient conditions for independence between $W_1$ and $W_2$. But I'm suspicious and I am asking you help to find a mistake.

Let $M_{W_1, W_2}$ denote the moment generating function of $W_1, W_2$. By construction $$ M_{W_1, W_2}(s,t)=\frac{1}{2}\exp(s\mu_{X,1}+t\mu_{X,2}+s^2v_{X,1}+t^2v_{X,2}) $$ $$ +\frac{1}{2}\exp(s\mu_{Y,1}+t\mu_{Y,2}+s^2v_{Y,1}+t^2v_{Y,2}) $$

Proposition: $M_{W_1, W_2}(s,t)=M_{W_1, W_2}(s,0)\times M_{W_1, W_2}(0,t)$ $\forall (s,t)\in \mathbb{R}^2$ if and only if $W_1$ independent of $W_2$

In my case $$ M_{W_1, W_2}(s,0)\times M_{W_1, W_2}(0,t)=\frac{1}{4}\exp(s\mu_{X,1}+t\mu_{X,2}+s^2v_{X,1}+t^2v_{X,2}) $$ $$ +\frac{1}{4}\exp(s\mu_{Y,1}+t\mu_{Y,2}+s^2v_{Y,1}+t^2v_{Y,2}) $$ $$ +\frac{1}{4}\exp(s\mu_{X,1}+t\mu_{Y,2}+s^2v_{X,1}+t^2v_{Y,2}) $$ $$ +\frac{1}{4}\exp(s\mu_{Y,1}+t\mu_{X,2}+s^2v_{Y,1}+t^2v_{X,2}) $$ Notice that $$ M_{W_1, W_2}(s,0)\times M_{W_1, W_2}(0,t)=M_{W_1, W_2}(s,t) $$ if and only if

$\mu_{X,1}=\mu_{Y,1}$ and $v_{X,1}=v_{Y,1}$

or

$\mu_{X,2}=\mu_{Y,2}$ and $v_{X,2}=v_{Y,2}$

Therefore, by the proposition above, $W_1 \perp W_2$ if and only if

$\mu_{X,1}=\mu_{Y,1}$ and $v_{X,1}=v_{Y,1}$

or

$\mu_{X,2}=\mu_{Y,2}$ and $v_{X,2}=v_{Y,2}$.

$\endgroup$
  • $\begingroup$ Are $X$ and $Y$ independent vectors? $\endgroup$ – Alecos Papadopoulos Nov 5 '17 at 16:10
  • 3
    $\begingroup$ I agree with your conclusion that the two components are independent iff the mixture is not a mixture wrt one of the two components. Otherwise, they are necessarily dependent. $\endgroup$ – Xi'an Nov 5 '17 at 16:13
3
+50
$\begingroup$

In an alternative way you could use the requirement and sufficient condition that you have independence iff the combined probability of $X_1$ and $X_2$ is the product of the probability of $X_1$ and $X_2$.

$$f_{(X_1,X_2)} = f_{X_1} f_{X_2}$$

And this you could work out more easily (more easily in the sense that the part after your 'notice that...' is not so trivial):

$W$ as mixture of $X$ and $Y$ $$f_{(W_1,W_2)} = a f_{(X_1,X_2)} + b f_{(Y_1,Y_2)}$$

$W$ as product of PDFs of mixture of $X_1$ and $Y_1$ and mixture of $X_2$ and $Y_2$ (only iff $W_1$ and $W_2$ are independent) $$f_{(W_1,W_2)} = \hphantom{\frac{1}{2}} f_{W_1}f_{W_2} \hphantom{+ \frac{1}{2} f_{(Y_1,Y_2)}}$$

Which leads to: $$a f_{X_1}f_{X_2} + b f_{Y_1}f_{Y_2} = (a f_{X_1} + b f_{Y_1})(a f_{X_2}+ b f_{Y_2})$$

And, for any $a+b=1$, you get after some algebra:

$$(f_{X_1}-f_{Y_1}) (f_{X_2}-f_{Y_2}) = 0$$

So either $$f_{X_1} = f_{Y_1} \qquad or \qquad f_{X_2} = f_{Y_2}$$

Note that this generalizes your results to other multivariate distributions, allows adding more variables e.g. if $f_{X_1} = f_{Y_1} = f_{Z_1}$, and allows to use different mixing ratios.


The algebra in steps:

starting from:

$$a f_{X_1}f_{X_2} + b f_{Y_1}f_{Y_2} = (a f_{X_1} + b f_{Y_1})(a f_{X_2}+ b f_{Y_2})$$

bring the terms on the right hand side to the left and group by $a f_{X_1}$ and $b f_{Y_1}$

$$a f_{X_1}(f_{X_2} - (a f_{X_2}+ b f_{Y_2}) ) + b f_{Y_1}(f_{Y_2}- (a f_{X_2}+ b f_{Y_2})) = 0$$

simplify using $a+b=1$

$$a f_{X_1}(bf_{X_2} - (b f_{Y_2}) ) + b f_{Y_1}(a f_{Y_2}- (a f_{X_2})) = 0$$

divide by $a$ and $b$

$$ f_{X_1}(f_{X_2} - f_{Y_2} ) + f_{Y_1}(f_{Y_2}- f_{X_2}) = 0$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.