3
$\begingroup$

I have two sets of data points and want to know the correlation. I.e., my Hypothesis is "The two set of data points are correlated".

The data points are (if at all) non-linear correlated. This is why I chose distance correlation as my metric.

However, I also want to know whether or not my results are significant, i.e., is the (non)correlation strong enough based on my data points that I can confidently reject my initial hypothesis.

What test do I need to apply to calculate the significance of a distance correlation value?

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

Looking at the original paper, I would recommend their approach (section 5):

To implement the distance covariance test for small samples, we obtain a reference distribution for $n\mathcal{V}^2_n$ under independence by conditioning on the observed sample, that is, by computing replicates of $n\mathcal{V}^2_n$ under random permutations of the indices of the Y sample.

Essentially, you have your current distance correlation $\mathcal{R}_0$. You randomly reallocate each observation from set 1 to a different observation from set 2, without replacement, and recalculate the distance correlation. Your P-value is the number of resamples where the distance correlation is greater than $\mathcal{R}_0$, so I would recommend doing at least on the order of 10,000 resamples assuming your dataset is not so small that you can actively exhaust all possible permutations.

By the way, this will only allow you to reject a hypothesis of no correlation, but that is usually what these kinds of tests do ('the two sets are correlated' is usually the alternative hypothesis).

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thank you for your response. To make sure I understand you correctly. Given the observations [(1,2),(3,4),(4,5)] I would do 10k permutations where I change up the first observation, e.g., [(3,2),(4,4),(1,5)] and then recalculate the distance correlation. I count the amount of times the correlation is higher and divide it by the amount of permutations? $\endgroup$
    – Sim
    Commented Dec 11, 2023 at 14:24
  • 1
    $\begingroup$ Yes, that sounds about right - obviously you wouldn't be able to make 10K unique permutations from just 3 pairs. Just keep one side of indices fixed and select a random match from the other set. $\endgroup$
    – PBulls
    Commented Dec 11, 2023 at 14:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.