3
$\begingroup$

I asked something similar at the math exchange, but not I see it fits much better here.
When creating random samples (independly) for each variable dimension, we should have no correlations. But if we do this for many dimensions d, and calculate the pairwise correlation cij, we will find that sometimes the magnitude of correlation (using the linear correlation factor) is quite significant, just by chance!
I wonder how large is e.g. the expected average or maximum correlation?
Example: You create a 3D set of independent uniform variables, so we can calculate 2D correlation between x1,2, x1,3 and x2,3, like c12=-0.1, c13=0, c23=+0.1 (or whatever), so the maximum correlation magnitude is 0.1 and the average magnitude is 0.066.
How do these two calculated values depend on point count N and number of variables d? The background is that I want to compare low-discrepancy samples (LDS) against random samples on correlation. Also LDS should have no correlations, but as most LDS methods are not perfect, there is of course some correlation. Also a result for normal variables (instead of uniform) would be helpful. Also the expectation of c*c (if this is easier to calculate).

Bye Stephan

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Comment. A surprisingly broad question.

This page shows simulation results for correlations of two independent standard uniform distributions. Results seem to differ greatly for various (especially small) numbers $n$ of pairs of points. Each histogram represents $100,000$ values of $r.$

enter image description here

For normal data, of course, there is Fisher's work on the distribution of $r.$ (See Wikipedia.)

When you say 'average correlation', do you mean $E(r)$ or $E(|r|)?$ When you say 'maximum correlation', what sample sizes do you have in mind to keep the maximum below 1? To start: I hope you can specify a few cases of greatest interest (dimensions, sample sizes, uniform vs. normal vs. what other distributions?).

Addendum per @whuber's Comment: Further experimentation. Five observations on each of five independent uniform variables.

set.seed(906)
n = 5;  m = 5;  u = runif(n*m)
DTA = matrix(u, nrow=m)
cor(DTA)
           [,1]       [,2]       [,3]        [,4]        [,5]
[1,]  1.0000000  0.1957409 -0.8233160 -0.39093048 -0.62212314
[2,]  0.1957409  1.0000000 -0.1255770  0.62306226 -0.61205725
[3,] -0.8233160 -0.1255770  1.0000000  0.36625813  0.22424637
[4,] -0.3909305  0.6230623  0.3662581  1.00000000  0.08519501
[5,] -0.6221231 -0.6120572  0.2242464  0.08519501  1.00000000
max(abs(cor(DTA)-diag(5)))
[1] 0.823316

Highest absolute correlation happens to be about $|r| \approx 0.82,$ between variables 1 and 3. Matrix plot shows all ${5 \choose 2}$ pairs---see center top row. (Max absolute correlations this high are not rare. Tried several runs before this one with set.seed to post.)

pairs(DTA)

enter image description here

Improve this answer
$\endgroup$
3
  • 2
    $\begingroup$ The interesting aspect of this question is that it concerns the maximum absolute correlation. That's not easily derivable from a study of pairwise correlations alone because the correlations in a multivariate dataset are (highly) interdependent. $\endgroup$
    – whuber ♦
    Commented Sep 5, 2019 at 22:36
  • 1
    $\begingroup$ The question was interesting, so I appreciated. So don’t misunderstand the following statement. When you write “the correlation can be quite significant” and then in the example you cite an average corr of 6% and a maximum of 10%. Well it is not that much, especially considering that you are simulating a finite number of samples of finite sizes and the independence works based on the expectations, so the maximum realization of 10% may be a very irrelevant thing here. $\endgroup$
    – Fr1
    Commented Sep 6, 2019 at 10:46
  • $\begingroup$ Thanks to BruceET et al!! Getting a uniform distribution for N=4 is amazing. My main interest is on multi-variate uniform (or normal) distributions, e.g. having 10 dimensions and moderate count N, like 256. The reason for this is that for such cases LDS still works, but I want to know how much it is better e.g. regarding a random set and max absolute correlation. $\endgroup$
    – user32038
    Commented Oct 8, 2019 at 12:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.