# Independence of multivariate normal distribution

Consider the multivariate normal distribution $X = [X_1, X_2, X_3]^T$ with

$$\mu = \begin{pmatrix} -3 \\ 1 \\ 4\end{pmatrix} \quad \quad \Sigma = \begin{pmatrix} 4 & -1 & 0\\ -1 & 5 & 0\\ 0 & 0 & 2\end{pmatrix}$$

I need to find out, if the following random variables are independent:

• $(X_1, X_2)$ and $X_3$
• $X_1 - X_2$ and $X_1 + X_2 - X_3$

How can I determine that? I see that e.g. $X_1$ and $X_3$ have covariance $= 0$. Normally, this doesn't imply, that they are independent, but (as I've learned from Wikipedia) since here $X_1$ and $X_2$ are normally and jointly distributed, it does in fact apply. But how do I cope with the sums and minuses?

• Do you know what the mean and variance of sums and differences of variables are, given their means, variances and covariances? i.e. what $Var(X+Y)$ is, for example? Do you know a fact about the distribution of linear combinations of jointly normal random variables? – Glen_b Sep 29 '13 at 14:35
• What @Glen_b is hinting at is the formula Var$(X+Y)$ = Var$(X)$ + Var$(Y)$ + Covar$(X,Y)$. Equivalently, the covariance of linear combinations $a X$ and $b X$, for $a=(a_1,a_2,a_3)$ and $b=(b_1,b_2,b_3)$, is given by $a\Sigma b'$. This makes short work of the question: just compute the values of two such expressions and compare them to zero. – whuber Sep 29 '13 at 17:27
• In addition to the clarification offered by @wuber, you also need to make use of the fact that linear combinations of jointly normal random variables are also normal. – Glen_b Sep 29 '13 at 22:54
• @Glen_b I've just added the self-study tag - sorry! I didn't know this tag existed. – kafman Sep 30 '13 at 20:32
• No worries, you can't know everything when you're just getting started. Thanks for adding the tag. [The help mentions the homework tag (which now maps to the self-study tag). It's worth having a look through the help if you haven't used SE sites much.] – Glen_b Oct 1 '13 at 0:45

Indeed, for the normal distribution, uncorrelatedness implies independence. For your first case, showing formally independence between the random vector $(X_1, X_2)$ and the scalar random variable $X_3$ can be done by showing that the conditional on $X_3$ mean and conditional covariance matrix of the random vector $(X_1, X_2)$ is equal to the unconditional mean and covariance matrix. Using notation as stated in this Wikipedia article,

$\Sigma$ is the $3\times 3$ covariance matrix of the joint distribution of your three variables. It is then unequally partitioned into sub-matrices. Denoting $v_{ij}$ the elements of $\Sigma$, we have

$$\Sigma_{11} =\left[\begin{matrix} v_{11} &v_{12} \\ v_{21} & v_{22} \end{matrix}\right] = \left[\begin{matrix} 4 &-1 \\ -1 & 5 \end{matrix}\right]$$

$$\Sigma_{12} =\left[\begin{matrix} v_{13} \\v_{23} \end{matrix}\right] = \left[\begin{matrix} 0 \\0 \end{matrix}\right] \qquad \Sigma_{21} = \Sigma_{12}'$$

$$\Sigma_{22} = v_{33} = \sigma_3^2 = 2$$

Then the conditional expectation vector- function $E\Big[(X_1,X_2)\mid X_3)\Big]$ is

$$E\Big[(X_1,X_2)\mid X_3)\Big] = \left [\begin{matrix} \mu_1 \\ \\\mu_2\end{matrix} \right] + \Sigma_{12}\Sigma^{-1}_{22}(X_3 - \mu_3)$$

which is equal to the unconditional mean-vector since $\Sigma_{12} = \mathbb 0$.The analogous result can be easily shown to hold for the covariance matrix.

As for your second question, one can proceed as follows:

Define $Y_1 = X_1 - X_2$ and $Y_2 = X_1 + X_2 - X_3$. The $Y$'s are still normally distributed random variables. So independence is here too equivalent to uncorrelatedness, i.e. zero covariance. So all you have to do is calculate

$$\operatorname{Cov}(Y_1,Y_2) = E(Y_1Y_2) - E(Y_1)E(Y_2)$$

and see whether it equals zero (normally, it won't).

• Thanks for the extensive answer! I was not aware of the fact, that showing that the unconditional mean (resp. covariance-matrix) equals the conditional mean (resp. covariance-matrix) is sufficient to prove the independence of two distributions. Why is that so? – kafman Sep 29 '13 at 17:10
• Moments characterize a distribution. For those distributions that are uniquely characterized by a few moments, then it is one of the approaches to check for independence. For the normal, it is the mean and variance. And you should state it in reverse: the conditional moment equals the unconditional. – Alecos Papadopoulos Sep 29 '13 at 17:29
• This seems like an awfully convoluted way to check independence, when all that is needed in each case is to compute a covariance. – whuber Sep 29 '13 at 17:30
• Alecos, I think a little more caution about completely solving the question is in order for what appears to be a self-study question (even if the OP hasn't yet tagged it that way). That is, it looks like you're 'doing the OP's homework', in detail. – Glen_b Sep 29 '13 at 22:59
• @Glen_b hmmm... see what you mean. Message received. – Alecos Papadopoulos Sep 29 '13 at 23:02