$X$ is multivariate normal distributed with $\mu = (2, 2)$ and $\Sigma = (1, 0 ; 0, 1)$ and I have 2 vectors $A = (1, 1)$ and $B = (1, -1)$. How can I than show that the linear combinations $AX$ and $BX$ are independent?

  • $\begingroup$ @dsaxton Not really, sure there is a relationship, but the questions were neither the same nor were the answers the same. $\endgroup$ – Carl Oct 7 '16 at 4:15
  • $\begingroup$ @dsaxton In fact, I don't like the answer to that other question. Proving the covariance is zero does not directly show only independence was ever assumed. It misses the full flavor of the absurdity of assuming dependence of an independent variable. $\endgroup$ – Carl Oct 7 '16 at 4:22
  • 1
    $\begingroup$ @Carl mathematically the other question is identical, except for the trivial difference that there the means of $X$ and $Y$ are 2 rather than 0. As for the other part of your comment, I am not sure what you mean. (Note that the result does depend on $X$ and $Y$ being Gaussian.) $\endgroup$ – GeoMatt22 Oct 7 '16 at 4:31
  • $\begingroup$ @GeoMatt22 Changed answer, at this point I do not understand what depends on $X$ and $Y$ being Gaussian-Is that relevant at this time? $\endgroup$ – Carl Oct 7 '16 at 18:46
  • 1
    $\begingroup$ @Carl I believe Dilip Sarwate has the right approach for the general case. For the particular case in this question, where the change of coordinates is isotropic (rotation + uniform scaling), I believe you can think of it visually like this: There are many stable distributions, but the (isotropic multivariate) Gaussian is the only separable joint PDF that is rotationally symmetric. (Note that closure under convolution $\neq$ closure under multiplication.) $\endgroup$ – GeoMatt22 Oct 7 '16 at 19:10

First of all independence follows from a lack of correlation when two random variables are jointly normal. Now it isn't hard to see that $AX$ and $BX$ are uncorrelated:

\begin{align} \text{Cov}(AX, BX) &= \text{Cov}(X_1 + X_2, X_1 - X_2) \\ &= \text{Var}(X_1) - \text{Var}(X_2) \\ &= 0 \end{align}

so we just need to show that $AX$ and $BX$ are jointly normal. To do that it's enough to show that all linear combinations of $AX$ and $BX$ are normally distributed. Can you finish the proof?


The definition of joint normality of $W$ and $Z$ is that all linear combinations $aW+bZ$ of $W$ and $Z$ are normal random variables. Since $X$ and $Y$ are jointly normal, this shows that $W=X+Y$ and $Z=X-Y$ are normal random variables. Furthermore, $aW+bZ = (a+b)X+(a-b)Y$ is normal because it is a linear combination of jointly normal random variables $X$ and $Y$. We conclude that $X+Y$ and $X-Y$ are jointly normal random variables. Hence they are independent if their covariance is $0$. I leave it to you to use the bilinearity of the covariance function to determine whether they are indeed independent random variables.

Look, Ma! No explicit formulas for pdfs used anywhere!

  • $\begingroup$ I think your proof is formally correct. I think you are assuming of convolution a property that it cannot have, and then showing that that property is not there. This is proof by disproof of the contrary, which may be unnecessary gymnastics. I would stop where you begin, by saying that convolution is an independent, orthogonal, process. $\endgroup$ – Carl Oct 7 '16 at 17:13
  • $\begingroup$ The proof has another interpretation. The covariance being zero shows that it is incorrect to assume covariance for convolution, which is the desired contradiction. $\endgroup$ – Carl Oct 7 '16 at 18:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.