I've tried to find an answer to this problem by searching online, but I couldn't, so please forgive me if this question has been asked and answered. :)

I have two vectors of length n1 and n2, respectively, and I would like to run a permutation test in order to understand whether their differential correlation is larger than zero (see Edit1 for details). To do so, I randomly permute elements from the first vector and the second vector.

My problem is to evaluate, given only n1 and n2, the number of elements that should be permuted at each permutation step. By doing some simulations (code posted and explained below), I noticed that this value follows a normal distribution N(μ, σ), as confiremed by the function fitdist in the package fitdistrplus. When n1=n2 μ is equal to n1/2 and σ is sqrt(n1*0.125). However, when n1 != n2 (I assume here that n2 > n1) things are more complicates. After some simulations, it seems to me that μ = n1 * (-0.604 * sqrt(n1/n2) + 1.101), but I can't find a way to calculate σ.

Does anyone know if there is an elegant way to define μ and σ from n1 and n2?

The following function has been used for simulation:

count.permuted.elements <- function(n1, n2, cycles=10000) {
    changed <- numeric(cycles)
    for (i in 1:cycles) {   
        v <-sample(c(rep(1, n1),rep(0, n2)))
        changed[i] <- n1 - sum(v[1:n1])


The function shuffles an array that contains n1 elements set to 1 and n2 elements set to 0, and then counts how many elements changed in the smaller vector (that is how many elements in the first n1 position of v are no longer set to 1).

Using the function above I generated the following histograms showing the distributions of the number of elements that changed when n1=50 and n2 is 50, 100, 200 and 400 (10000 permutation performed). Of course the right tail is limited by n1 (see plot in the bottom right).

enter image description here

EDIT1: I am doing a test on differential correlation, that is I have two paired vectors (let's say eight and weight measurements), measured in two independent sets (let's say male and female) and the number of elements in each set may be different (n1 being female and n2 being male). The correlation in females is rhoF and in males is rhoM, and I would like to understand if the null hypothesis that rhoF = rhoM can be rejected. To do so, I need to move elements (paired) between sets in order to remove the relationships between measurements and sets and then recompute the correlation rhoF and rhoM each time, counting how many times their difference is at least as extreme as the one evaluated on the original data.

  • 1
    $\begingroup$ This is a bit confusing. You're doing a test about a correlation coefficient so you should have paired data, in which case $n_1 = n_2$. I'm also not sure what's unclear about shuffling the data. You just reorder them randomly and recompute the correlation each time. $\endgroup$ – dsaxton Feb 29 '16 at 18:54
  • 1
    $\begingroup$ Could you explain what a "correlation difference" is? Also, it's unclear why you are asking about the number of elements: by definition, a permutation potentially affects all indexes of the data. How many indexes are fixed by any such permutation is a random variable. $\endgroup$ – whuber Feb 29 '16 at 19:20
  • $\begingroup$ According to the edit, a "differential correlation" is difference between correlation coefficients. The null hypothesis therefore is that the two groups have equal correlations. That determines what you can, and cannot, do in a permutation test. It's not at all clear how the code fragments you present are consistent with that null hypothesis. I do not see any recomputation of correlations in that code. Regardless, it seems you want to swap values among the two groups, but that would not be a valid way to represent the null distribution: this test just won't work. $\endgroup$ – whuber Mar 1 '16 at 14:05
  • $\begingroup$ In fact, the function doesn't calculate differential correlation, but counts how many elements are swapped between the two conditions at each permutation step. I know that this number follows a normal distribution, and I would like to estimate its parameter --I must admit that the application to differential correlation is not very relevant here. $\endgroup$ – alesssia Mar 1 '16 at 17:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.