I've been trying to implement the iman-conover method in python so I could generate correlated random numbers from distributions other than normal (I use a normal & uniform in my example below).
There are a few functions (modified from the first blog):
import numpy as np from scipy import stats import matplotlib.pyplot as plt def ic_m(n, d): a = np.arange(1, (n+1)) p = stats.norm.ppf(a/(n+1)) p = normalize(p) score = np.zeros((n, d)) for j in range(0, score.shape): score[:, j] = np.random.permutation(p) return score def normalize(v): norm=np.linalg.norm(v, ord=1) return v/norm def rank(N): rank = np.zeros((N.shape, N.shape)) for j in range(0, N.shape): rank[:, j] = stats.rankdata(N[:, j], method='ordinal') return rank.astype(int) - 1 def reorder(rank, samples): rank_samples = np.zeros((samples.shape, samples.shape)) for j in range(0, samples.shape): s = np.sort(samples[:, j]) rank_samples[:, j] = s[rank[:,j]] return rank_samples
And then the actual test of the method:
n, d = 1000, 2 corrTar = .2 S = np.array(([1., corrTar], [corrTar, 1.])) C = np.linalg.cholesky(S) M = ic_m(n,d) D = (1./n) * np.dot(M.T, M) E = np.linalg.cholesky(D) N = np.dot(np.dot(M, np.linalg.inv(E)), C) R = rank(N) dists = np.array(( stats.norm.ppf(np.random.uniform(0.0, 1.0, n), loc=0, scale=1), stats.uniform.ppf(np.random.uniform(0.0, 1.0, n), 0, 1) )) dists = reorder(R, dists.T) np.corrcoef(dists.T) x =dists.T y =dists.T
This generates roughly what I'd expect when creating a scatter plot of the data (I'd post images but am limited to 2 links). If I iterate through that process 1,000 times and record the spearman correlation after each test I get a distribution that is centered around the desired correlation.
The problem occurs when I increase the desired correlation to anything above 0.6 (0.85 as an example). The resulting correlation is centered around 0.60 and never even approaches the 0.85. I do not believe this is normal behavior and haven't been able to see where I misrepresented the method in my code.
Can anyone see what I cannot?