I have empirically noticed and interesting phenomenon. Suppose we have two continuous random variables $X$ and $Y$ which are dependent but not correlated. For instance:

$X \sim \mathcal{N(0,1)}$

$Y = cos(X) +Z$,$~~~~$ where $Z\sim \mathcal{N(0,0.5)}$

You can use the following Python code to generate samples for these variables and check that they are indeed dependent but not correlated (I use this implementation of distance correlation as a measure for dependence):

N_SAMPLES = 10000

def genSamples(n):

    x = np.random.normal(0,2,size=(n))
    y = np.cos(x)+np.random.normal(0,0.5, n)

    return x,y

x,y = genSamples(N_SAMPLES)


print "CorrCoef: ", np.corrcoef(x,y)[0,1]
print "DistCorr: ", distcorr(data[:,0], data[:,1])

Now consider two additional random variables $X_{sum}$ and $Y_{sum}$ generated as the summation of $n$ realizations of the original variables. The thing I have noticed is that $X_{sum}$ and $Y_{sum}$ become more and more independent as $n$ grows.

N_SUM = 1

Xsum = np.zeros((N_SAMPLES))
Ysum = np.zeros((N_SAMPLES))

for _ in range(N_SUM):

    x,y = genSamples(N_SAMPLES)

    Xsum += x
    Ysum += y


print np.corrcoef(Xsum,Ysum)[0,1]
print distcorr(Xsum, Ysum)

enter image description here

I know that by the Central Limit Theorem $X_{sum}$ and $Y_{sum}$ will be approximately normal, but why are they becoming independent? Is there a general explanation for this?. Also, I have noticed that this phenomenon only occurs if $X$ and $Y$ are not correlated.

Thanks in advance.

  • $\begingroup$ Can you please give a mathematical definition of $X_{sum}$ and $Y_{sum}$? My R results do not match your Python results. Stating in clear mathematical terms would help those of us who do not use Python regularly or do not have access to Python. $\endgroup$ – EliK Jul 26 '18 at 14:30
  • $\begingroup$ When I run my own simulations I get that $X_{sum}$ and $Y_{sum}$ are highly correlated. We must be having different problems. Again, a mathematical explanation of what $X_{sum}$ and $Y_{sum}$ would help a lot. $\endgroup$ – EliK Jul 26 '18 at 15:09
  • $\begingroup$ $X_{sum}$ is the summation of $n$ independent realizations of the random variable $X_{sum}$, and $Y_{sum}$ is the summation of $n$ realizations of the random variable $Y$ based on the n realizations of the variable $X$. The code is easy to understand! P.S. Make sure you use the $\sigma$ values I suggested, otherwise $X$ and $Y$ will likely be correlated. $\endgroup$ – Daniel López Jul 26 '18 at 19:32

I was able to figure out the answer with the help of the folks at Mathematics Stack Exchange.

Consider the random vector $[X_{sum}$, $Y_{sum}]^\top$. Since this vector is the sum of $n$ i.i.d. random vectors, the multidimensional CLT applies and we can ensure that

$$\sqrt{n} \begin{bmatrix} \frac{1}{n} X_{sum} - \mathbb{E} X \\ \frac{1}{n} Y_{sum} - \mathbb{E} Y \end{bmatrix} \overset{\mathcal{D}}{\rightarrow} \mathcal{N}_2(0,\Sigma)$$

where the covariance matrix $\Sigma$ is:

$$\Sigma=\begin{bmatrix}\text{Var}(X) & \text{Cov}(X,Y) \\ \text{Cov}(X,Y) & \text{Var}(Y)\end{bmatrix}=\begin{bmatrix}\text{Var}(X) & 0 \\ 0 & \text{Var}(Y)\end{bmatrix}$$ since $X$ and $Y$ are uncorrelated. So we have a bivariate normal with diagonal corvariance, and because of this its two variables have no option but to be independent [1].

Also note that we are interested in $[X_{sum}$, $Y_{sum}]^\top$; but this is the same as the above vector multiplied by a constant and translated, so the independence of the variables that form it is not affected.

[1] Robert V. Hogg, Joseph W. McKean, and Allen T. Craig. "Introduction to Mathematical Statistics, 7th Edition". In: Pearson, 2013, pp. 182-183. isbn: 978-0-321-84943-4.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.