Two variables that are uncorrelated are not necessarily independent, as is simply exemplified by the fact that $X$ and $X^2$ are uncorrelated but not independent. However, two variables that are uncorrelated AND jointly normally distributed are guaranteed to be independent. Can someone explain intuitively why this is true? What exactly does joint normality of two variables add to the knowledge of zero correlation between two variables, which leads us to conclude that these two variables MUST be independent?
$\begingroup$
$\endgroup$
12
asked Nov 9, 2018 at 19:27
2 Answers
$\begingroup$
$\endgroup$
13
The the joint probability density function (pdf) of bivariate normal distribution is: $$f(x_1,x_2)=\frac 1{2\pi\sigma_1\sigma_2\sqrt{1-\rho^2}}\exp[-\frac z{2(1-\rho^2)}], $$
where
$$z=\frac{(x_1-\mu_1)^2}{\sigma_1^2}-\frac{2\rho(x_1-\mu_1)(x_2-\mu_2)}{\sigma_1\sigma_2}+\frac{(x_2-\mu_2)^2}{\sigma_2^2}.$$ When $\rho = 0$, $$\begin{align}f(x_1,x_2) &=\frac 1{2\pi\sigma_1\sigma_2}\exp[-\frac 12\left\{\frac{(x_1-\mu_1)^2}{\sigma_1^2}+\frac{(x_2-\mu_2)^2}{\sigma_2^2}\right\} ]\\ & = \frac 1{\sqrt{2\pi}\sigma_1}\exp[-\frac 12\left\{\frac{(x_1-\mu_1)^2}{\sigma_1^2}\right\}] \frac 1{\sqrt{2\pi}\sigma_2}\exp[-\frac 12\left\{\frac{(x_2-\mu_2)^2}{\sigma_2^2}\right\}]\\ &= f(x_1)f(x_2)\end{align}$$.
So they are independent.
answered Nov 9, 2018 at 20:51
-
-
1$\begingroup$ Thank you all. Elegant proof. It's clear now. It seems to me that given the flow of the proof, I should have asked what knowledge of zero correlation adds to knowledge of joint normality and not the other way around. $\endgroup$ Nov 9, 2018 at 21:18
-
1$\begingroup$ What about an intuitive explanation as to why it is true? $\endgroup$ Nov 10, 2018 at 12:51
-
$\begingroup$ Maybe there is no intuitive simple explanation. $\endgroup$ Nov 10, 2018 at 15:34
-
$\begingroup$ Could we get some intuition along the line of reasoning that a Gaussian process' higher (than 2) moments are all zero, and adding the zero correlation condition (moment 2), pins down all moments greater than one to zero? $\endgroup$ Nov 11, 2018 at 13:33
$\begingroup$
$\endgroup$
Joint normality of two random variables $X,Y$ can be characterized in either of two simple ways:
For every pair $a,b$ of (non-random) real numbers, $aX+bY$ has a univariate normal distribution.
There are random variables $Z_1,Z_2\sim\operatorname{\text{i.i.d.}} \operatorname N(0,1)$ and real numbers $a,b,c,d$ such that $$\begin{align} X & = aZ_1+bZ_2 \\ \text{and } Y & = cZ_1 + dZ_2. \end{align}$$
That the first of these follows from the second is easy to show. That the second follows from the first takes more work, and maybe I'll post on it soon . . .
If the second one it true, then $\operatorname{cov}(X,Y) = ac + bd.$
If this covariance is $0,$ then the vectors $(a,b),$ $(c,d)$ are orthogonal to each other. Then $X$ is a scalar multiple of the orthogonal projection of $(Z_1,Z_2)$ onto $(a,b)$ and $Y$ onto $(c,d).$
Now conjoin the fact of orthogonality with the circular symmetry of the joint density of $(Z_1,Z_2),$ to see that the distribution of $(X,Y)$ should be the same as the distribution of two random variables, one of which is a scalar multiple of the orthogonal projection of $(Z_1,Z_2)$ onto the $x$-axis, i.e. it is a scalar multiple of $Z_1,$ and the other is similarly a scalar multiple of $Z_2.$
answered Nov 9, 2018 at 21:41
X <- rnorm(n=10000); X2 <- X*X; cor(X[X>1],X2[X>1])$\endgroup$