# p-value adjustment via permutation for multiple correlations

I have the following problem: I have performed 3200 correlations between a certain measure of connectivity and 1 behavioural outcome in a sample of subjects. As the single correlations (represented by R and p) are not independent from each other, Bonferroni or FDR correction (at given p-values) failed to leave any significant correlations.

So I tried to follow this permuation approach: While permutating the available subject's connectivity data on the one hand, and also permutating the behavioural data on the other , I created a new pseudo-sample (in which subjects connectivity and behaviour are not matching anymore) for which I calculated all correlations again (3200). I did this 10000x and thus generated 10000x3200 random p-values based on the actual data set. Having ordered all p-values (low-to-high) I looked at the lowest 5% of those p-values (alpha): Random correlations would reach a p-value of 0.01 by 5% chance. In another group, the 5% lowest p-value would only reach 0.052.

My question would be whether this would be a plausible way to correct for multiple correlations in a dataset with lots of non-independent single correlations. Particularly, is it reasonable to use an adjusted p-value that is even higher than the original one, 0.05. Maybe the chance to reach 0.05 on that data set is even lower than 5% due to variance etc.

Thank you very much

Best, Robert

-

Westfall and Young have proposed resampling approaches to p-value adjustent which are applied in SAS using the MULTTEST procedure. Permmutation and bootstrap methods are used but may be different from what you are doing. You can check the SAS users guide for the MULTTEST procedure or read their book. Here is an amazon link to their book. http://www.amazon.com/Resampling-Based-Multiple-Testing-Adjustment-Probability/dp/0471557617/ref=sr_1_1?s=books&ie=UTF8&qid=1340734744&sr=1-1&keywords=Westfall+and+Young

-