1
$\begingroup$

I have many variables ( ~2000 and 320 samples) and I have found the correlation coefficient for each unqiue pair of these variables through Spearmann (underlying data does not have normal distribution). As some outliers can affect the correlation coefficient value, I am considering bootstrapping and/or permutation testing to be able to better understand whether there is any significant difference betweeen the original coefficient value and the one generated through a resampling method. I am unsure of a few things;

1)I realise (without resampling methods) that my null hypothesis would be that when considering a pair of variables X and Y , H0:ρ(X,Y)=0, whilst alternative H1:ρ(X,Y) ≠ 0. But following the resampling, my null hypothesis is that there is no difference between the original coefficient value and the one generated from resampling? As ultimately, this is what I need to test.

2)as both approaches are quite time intensive due to the number of variables involved - would one approach be more statistically valid than the other?

3)In addition, lastly, if I were to go with a bootstrapping approach and calculated 95% confidence intervals (CI), based on my null hypothesis above following resampling, if this interval contained 0, I presume this is indicative that there is no significant difference between my original and bootstrapped statistic?

Please could anybody help clarify?

$\endgroup$
5
  • $\begingroup$ Do you mean that you've computed the rank correlations for all ${2000 \choose 2} \approx 2,000,000$ pairs of $2,000$ variables. These on a sample of size $320$? $\endgroup$
    – dipetkov
    Mar 17, 2023 at 20:53
  • $\begingroup$ @dipetkov yes, that's correct. I will be transferring my analysis to a high performance computing cluster (using R within this), as even using a 10 core cluster within R on my 11 core laptop, this takes 12 hours (before permuting or bootstrapping). $\endgroup$
    – aim6789
    Mar 18, 2023 at 11:46
  • $\begingroup$ Even though the computations can be accomplished, I would be asking myself what, if anything, I would learn from 95%-level confidence intervals when there are 2 million of them. What is the purpose of these calculations? How are you going to control the type I error? $\endgroup$
    – dipetkov
    Mar 18, 2023 at 12:32
  • $\begingroup$ @dipetkov that's a good point. I guess it comes back to the discussion around p values vs confidence intervals (e.g. towardsdatascience.com/…), but perhaps calculating p values and correcting for multiple testing (i.e Bonferroni or FDR) might be the way forward. $\endgroup$
    – aim6789
    Mar 18, 2023 at 16:04
  • $\begingroup$ With this number of comparisons, I would say FDR makes more sense. $\endgroup$
    – dipetkov
    Mar 18, 2023 at 16:13

1 Answer 1

1
$\begingroup$

you don't require normal data to calculate pearson correlation.

I would suggest you instead calculate pearson correlation and use Fisher transformation to do your statistical test.

test the fisher transformation (which assumes normally distributed variables) using your bootstrapping approaches (on a subset of the pairs)

see this article https://www.uvm.edu/~statdhtx/StatPages/Randomization%20Tests/BootstCorr/bootstrapping_correlations.html which discusses 1) and confirms 3)

note that there is an equivalent of fisher transformation for spearman https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient - see determining significance

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.