permutation test on correlation coefficient

Question

I have a correlation coefficient of two hierarchical clustering trees with 20 labels.

I want to check the hypothesis that the labels of the trees are uncorrelated with a permutation test.

I calculated a p-value using 500 permutations

obsereved.coeff = 0.292

sample.permut <- c()
n <- 500

for(i in 1:n){
  t = tree2 %>% dist(method = "euclidean") %>% hclust("complete") %>% as.dendrogram %>% set("labels",permute(row.names(tree2)))
  sample.permut[i] = cor_bakers_gamma(tree1, t)
}
p-val = sum(abs(sample.permut)>=obsereved.coeff)/500
# p-value:  0.002

• are 500 permutations enough?
• is my p-value calculation correct?
• can I conclude that the labels of the two trees are related, with a 5% probability that the null is correct?

Answer 1

are 500 permutations enough?

Never write simulation code with 500 in it. Always write n <- 500 and use n for the rest. Then as your last step try increasing n to higher values. Only if that does not change the result in relevant ways assume that n was large enough. Or run it with n <-500 many times and see if the results are stable enough.

is my p-value calculation correct?

boils down as to whether everything is correct with p-val = sum(abs(sample.permut)>=obsereved.coeff)/500 As we do not know what sample.permut is and have not seen the rest of the code, we could not honestly say "yes, that is correct". Btw, have you considered one-tailed vs two-tailed testing?

with a 5% probability that the null is correct?

No. That is not what a $p$-value is.