Consider the multivariate normal distribution $X = [X_1, X_2, X_3]^T$ with

$$ \mu = \begin{pmatrix} -3 \\ 1 \\ 4\end{pmatrix} \quad \quad \Sigma = \begin{pmatrix} 4 & -1 & 0\\ -1 & 5 & 0\\ 0 & 0 & 2\end{pmatrix}$$

I need to find out, if the following random variables are independent:

  • $(X_1, X_2)$ and $X_3$
  • $X_1 - X_2$ and $X_1 + X_2 - X_3$

How can I determine that? I see that e.g. $X_1$ and $X_3$ have covariance $= 0$. Normally, this doesn't imply, that they are independent, but (as I've learned from Wikipedia) since here $X_1$ and $X_2$ are normally and jointly distributed, it does in fact apply. But how do I cope with the sums and minuses?

  • 3
    $\begingroup$ Do you know what the mean and variance of sums and differences of variables are, given their means, variances and covariances? i.e. what $Var(X+Y)$ is, for example? Do you know a fact about the distribution of linear combinations of jointly normal random variables? $\endgroup$ – Glen_b Sep 29 '13 at 14:35
  • 2
    $\begingroup$ What @Glen_b is hinting at is the formula Var$(X+Y)$ = Var$(X)$ + Var$(Y)$ + Covar$(X,Y)$. Equivalently, the covariance of linear combinations $a X$ and $b X$, for $a=(a_1,a_2,a_3)$ and $b=(b_1,b_2,b_3)$, is given by $a\Sigma b'$. This makes short work of the question: just compute the values of two such expressions and compare them to zero. $\endgroup$ – whuber Sep 29 '13 at 17:27
  • 1
    $\begingroup$ In addition to the clarification offered by @wuber, you also need to make use of the fact that linear combinations of jointly normal random variables are also normal. $\endgroup$ – Glen_b Sep 29 '13 at 22:54
  • 1
    $\begingroup$ @Glen_b I've just added the self-study tag - sorry! I didn't know this tag existed. $\endgroup$ – kafman Sep 30 '13 at 20:32
  • 1
    $\begingroup$ No worries, you can't know everything when you're just getting started. Thanks for adding the tag. [The help mentions the homework tag (which now maps to the self-study tag). It's worth having a look through the help if you haven't used SE sites much.] $\endgroup$ – Glen_b Oct 1 '13 at 0:45

Indeed, for the normal distribution, uncorrelatedness implies independence. For your first case, showing formally independence between the random vector $(X_1, X_2)$ and the scalar random variable $X_3$ can be done by showing that the conditional on $X_3$ mean and conditional covariance matrix of the random vector $(X_1, X_2)$ is equal to the unconditional mean and covariance matrix. Using notation as stated in this Wikipedia article,

$\Sigma$ is the $3\times 3$ covariance matrix of the joint distribution of your three variables. It is then unequally partitioned into sub-matrices. Denoting $v_{ij}$ the elements of $\Sigma$, we have

$$\Sigma_{11} =\left[\begin{matrix} v_{11} &v_{12} \\ v_{21} & v_{22} \end{matrix}\right] = \left[\begin{matrix} 4 &-1 \\ -1 & 5 \end{matrix}\right] $$

$$\Sigma_{12} =\left[\begin{matrix} v_{13} \\v_{23} \end{matrix}\right] = \left[\begin{matrix} 0 \\0 \end{matrix}\right] \qquad \Sigma_{21} = \Sigma_{12}'$$

$$\Sigma_{22} = v_{33} = \sigma_3^2 = 2 $$

Then the conditional expectation vector- function $E\Big[(X_1,X_2)\mid X_3)\Big]$ is

$$E\Big[(X_1,X_2)\mid X_3)\Big] = \left [\begin{matrix} \mu_1 \\ \\\mu_2\end{matrix} \right] + \Sigma_{12}\Sigma^{-1}_{22}(X_3 - \mu_3) $$

which is equal to the unconditional mean-vector since $\Sigma_{12} = \mathbb 0$.The analogous result can be easily shown to hold for the covariance matrix.

As for your second question, one can proceed as follows:

Define $Y_1 = X_1 - X_2$ and $Y_2 = X_1 + X_2 - X_3$. The $Y$'s are still normally distributed random variables. So independence is here too equivalent to uncorrelatedness, i.e. zero covariance. So all you have to do is calculate

$$\operatorname{Cov}(Y_1,Y_2) = E(Y_1Y_2) - E(Y_1)E(Y_2)$$

and see whether it equals zero (normally, it won't).

  • $\begingroup$ Thanks for the extensive answer! I was not aware of the fact, that showing that the unconditional mean (resp. covariance-matrix) equals the conditional mean (resp. covariance-matrix) is sufficient to prove the independence of two distributions. Why is that so? $\endgroup$ – kafman Sep 29 '13 at 17:10
  • 1
    $\begingroup$ Moments characterize a distribution. For those distributions that are uniquely characterized by a few moments, then it is one of the approaches to check for independence. For the normal, it is the mean and variance. And you should state it in reverse: the conditional moment equals the unconditional. $\endgroup$ – Alecos Papadopoulos Sep 29 '13 at 17:29
  • 2
    $\begingroup$ This seems like an awfully convoluted way to check independence, when all that is needed in each case is to compute a covariance. $\endgroup$ – whuber Sep 29 '13 at 17:30
  • 4
    $\begingroup$ Alecos, I think a little more caution about completely solving the question is in order for what appears to be a self-study question (even if the OP hasn't yet tagged it that way). That is, it looks like you're 'doing the OP's homework', in detail. $\endgroup$ – Glen_b Sep 29 '13 at 22:59
  • 1
    $\begingroup$ @Glen_b hmmm... see what you mean. Message received. $\endgroup$ – Alecos Papadopoulos Sep 29 '13 at 23:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.